Problem

According to The Washington Post [1], on average, you will need to wait 18.5 days before you can get an appointment to see your physician. A lot can happen during 18 days. Eventually, according to the Annals of Family Medicine, 18 percent of patients will decide to skip their appointments during this period because they are [2]:

• Feeling worse and need to go to the emergency room
  Overscheduled and forget their appointment
• Limited in their healthcare literacy and don’t understand or appreciate why the appointment is necessary
• Not in an established relationship with their doctor and aren’t concerned about missing an appointment
• Influenced by language barriers or, socio-economic factors and misunderstand when their appointment is

No show’s financial impact on the US healthcare system is estimated as $330+ billion year, resulting in US’s GDP reduction (2% of GDP).

Example – A doctor is supposed to see 15 patients every day – 10% no-show rate (*1) = 1.5 missed appointments daily = 8 no-shows per week (*2) – The doctor organizes appointments into 30-minute sessions at a cost of $100/session (*3). – Because of the 10% no-show rate, he loses $800 per week. This no-show rate costs the practice around $41,600 per year. – There are approximately 810,000 physicians in the U.S (*4). The total loss is estimated as $337 billion a year (*1) We assumed a conservative 10% no-show rate referring to this study [3] (*2) This example assumes a 5 day work week and does not exclude holidays (*3) We assumed a conservative $150 no-show cost referring to the same study above [3] (*4) Statista data [4]

Solution

We propose BookMe, a machine learning-based patient management tool (SaaS). With BookMe, medical providers can easily integrate their own website and BookMe Scheduler, which asks patients for an easy sign up and suggests available timeslots based on predicted no-show rates. BookMe also sends automated reminder texts and calls to those with relatively high no-show rates to minimize the costs associated with no show. BookMe predicts no-show rates analyzing a country-wide healthcare data, provider-specific data (if provided), transportation data etc. In addition to this 2-step no-show reduction process, each healthcare provider will get several relevant reports: capacity utilization report and BookMe performance report (prediction power, no-show reduction results, and additional revenue captured by this system).

If a clinic with 10 physicians can reduce no-show rate by 5% (half), it can save $208K a year. Clinics will expect cost cutting of employment through making scheduling and reminder operations automatic. BookMe costs free for small providers (up to 300 reservation a month) and $20/mo for larger providers (300+ reservation a month), both being attached with No-show Reduction Program (text/call reminder service).

Prototype Development Design

Data Collection

We would collect 10,000+ actual appointment data (gender, age, symptom, phone #, email address, and zip code) from our partner medical centers (University of Chicago and Rush University) through BookMe (beta) in combination with patient personal data (nationality, preferable language, family data, past diagnosis and prescription, etc) the medical centers own. From third party service providers, we would collect transportation data which would be related to one of the factors affecting the decision whether patients should skip their appointments or not. There are some preceding studies [5] about no-show patterns from which we could obtain insights for this process.

Model Development

We will structure Lasso-based logistic regression model with a minimum cross-validation error to predict accurate no-show rate (the higher the worse), which will be repetitively done whenever additional data is imported.

Model Validation

We would review the validation of the future appointment prediction, sorting out multiple reduction factors, in other words, whether the actual reduction comes from 1) optimized slot allocation or 2) no-show reduction program. Additionally, we would study on the effectiveness of adding provider-specific data to sample data because this affects how much data BookMe should collect from providers’ existing database in addition to the patients’ sign up data.

 

Team members:

Nobuhiro Kawai
Nadia Marston
Lisa Clarke-Wilson
Antonio Salomon

Source:

1. https://www.washingtonpost.com/news/wonk/wp/2014/01/29/in-cities-the-average-doctor-wait-time-is-18-5-days/
2. http://www.annfammed.org/content/2/6/541.full
3. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4714455/
4. https://www.statista.com/topics/1244/physicians/
5. http://www.mdpi.com/2227-9032/4/1/15/pdf

The Problem

Everyone’s familiar with class-action lawsuits where a bunch of families sue a pharmaceutical company. However, the same problem often happens for investors as well, leading to a securities class-action lawsuit. Essentially, if a company neglects its fiduciary responsibility to keep investors informed about negative changes in the company, and those eventually impact the company’s stock price when the news becomes public, investors are entitled to sue a company.

Right now, the existing solution is throwing bodies at the problem – plaintiff firms keep tons of lawyers on staff whose job is to read the news and track stocks, and hopefully identify a situation where a securities class-action lawsuit could be filed. This is incredibly manual and time-intensive, and is an impossible process to ensure success – a person is always going to miss some opportunities.

The Solution

Instead of relying on plaintiff lawyers and industry blogs, like Lyle Roger’s The 10b-5 Daily, to just manually scan and analyze stock price data, we believe there is an opportunity to merge human understanding and machine learning to identify and even predict potential security class action suits. To solve this problem we propose the creation of Classy, a service that predicts potential lawsuits for plaintiff lawyers by combining machine algorithms and human intuition. Currently many of the class action suits brought by firms end up being frivolous and yield limited to no profit for plaintiffs. Classy will help plaintiff firms to mediate this error and increase their efficiency in pursuing the most fruitful cases. Additionally Classy will help plaintiff firms better prioritize their staffing structure, so that there are more lawyers using their time to execute suits rather than searching for potential signs of fraud.  

The Design

Our product would combine external sensors with machine and human algorithms to predict the likelihood of securities misconduct of various firms and help analyze the success of a suit. The two sensory inputs would be stock prices and news articles. First, we would utilize machine learning to flag precipitous stock price drops throughout the whole market. We would also use natural language processing and sentiment analysis to analyze relevant news items, identifying patterns of negative disclosures by a firm in the past or public apologies issued by CEOs. These sensory inputs would then be analyzed by a machine algorithm, which would use the data to create a likelihood score of disclosure malfeasance by the firm and the predicted settlement value. This information would then be transmitted through a human algorithm – plaintiff lawyers with years of experience and relationship expertise – who would then verify and expand upon the potential suits flagged by the machine algorithm. They would also provide feedback to the machine algorithm in order to improve its efficacy and accuracy over time.

The Problem

The progresses in Artificial Intelligence (AI) in recent years, from data mining to computer vision and from natural language processing to robotics, have demanded people to start think about morality’s importance and complexity in the design of AI. According to the Economist almost half of all jobs could be automated by computers within two decades.[i] Many of these jobs are complex and involve judgements that lead to significant consequences.

The dilemma self-driving cars faces is a good example. In the situation of a brake failure, a self-driving car can either keep going straight, which would result in the death of pedestrians, or swerving to avoid hitting the pedestrians, which would result in the death of dogs crossing the street. How should the AI be programed to make decisions for the self-driving car in situations like this? Drones used by military to target and suppress terrorists is another well-debated example of the importance of morality in the design of AI. According to the New York Times, the Pentagon has put AI at the center of its strategy to maintain the United States’ position as the world’s dominant military power.^[ii] The new weapons would offer speed and precision unmatched by any human while reducing the number — and cost — of soldiers and pilots exposed to potential death and dismemberment in battle. How do we make sure these drones make the moral decision in the battlefield?

As innovation in AI accelerates, we need to get ahead of the curve and implements morality into the design of AI so that gains today are not taken at the cost of future abatement. We should define morality before an AI does.

Potential Challenges

The major challenge of programing ethics into AI is the fact that human ethical standards are currently imperfectly codified in law and they make all kinds of assumptions that are difficult to make. Machine ethics can be corrupted, even by programmers with the best of intentions. For example, the algorithm operating a self-driving car can be programed to adjust the buffering space it assigns to pedestrians in different districts based on monetary amount of settlement of previous accidents in each district. The assumption is that the bigger buffering space in districts with higher settlement costs can reduce the potential for higher settlement. The assumption seems reasonable, but it is possible that the lower settlements in certain districts were due to the lack of access to legal resources for residents of poorer neighborhood. Therefore, the algorithm could potentially disadvantage these residents based on their income.

The Solution

We have established the need to teach AI to have a learned concept of morality. However, given the challenges, governments and regulators need to be lead the effort to establish a globalized standard for machine ethics. These standards need to be clearly instituted and codified by legislature. However, governments’ lack the resources and talent in the field of AI would require them to have private sector’s involvement. At the same time, companies that are already developing products using AI such as self-driving cars have conflict of interests in assisting the government. This poses a potential opportunity for us. We can build a product that crowd-sources human opinions on how machines should make decisions when faced with moral dilemmas to help governments write these legislatures. For example, we could crowd source scenarios and the most appropriate responses to those scenarios for self-driving cars on our platform, and then contract with the governments to evaluate our findings and implement them into algorithms of self-driving cars looking to enter the market. Although sales process to government entities can be lengthy, our role as a third party between the regulator and the companies developing AI products and the potential multi-year revenue stream with a mandated project place us in a very good position.

[i] http://www.economist.com/news/briefing/21594264-previous-technological-innovation-has-always-delivered-more-long-run-employment-not-less

[ii] https://www.nytimes.com/2016/10/26/us/pentagon-artificial-intelligence-terminator.html

 

Team Awesome Members:

Rachel Chamberlain

Joseph Gnanapragasam

Cen Qian

Allison Weil

“The average worker today stays at each their job for 4.4 years, according to data from the Bureau of Labor Statistics, but the expected tenure of the youngest employees is half that.” – Forbes 2016

TalentSafe leverages signals from all over the workplace to predict who is at risk of attrition and how to maintain a healthy team.

The break up:

Quitting a job or firing someone is gruelling. However, not all turnover is bad. High employee-retention rate can be evidence of productivity, or alternatively suggest a culture of entitlement or one that fails to challenge employees. There is a healthy, ideal turnover rate applicable to different organizations.

The problem is not small. The potential benefits are massive.  

The real problem is how to increase the retention rate of high performance employees and maintain healthy retention rate of other employees.  

Solution:

Accurate predictions enable organizations to take action for retention or succession planning of employees. To solve this problem, organizations can use academic research backed machine learning techniques[2] to predict employee turnover. However, no solution exists in the market because of a few gaps that can be addressed.

Fix data gaps:

Modeling data, a big issue, comes from HR Information Systems (HRIS), which are typically underfunded compared to other Information Systems in the organization. This creates noise in the data that renders predictive models prone to inaccuracies. Studies[4] use Extreme Gradient Boosting (XGBoost) to improve predictions from the data.

TalentSafe can use data and signals from different sources, such as:

• Baselines from interview process and past experience, using studies[5]
  • Biodata
    • Employee reference
    • Prior job length
  • General work-related attitudes
    • Self-confidence
    • Decisiveness
    • Perseverance
  • Job-specific attitudes
    • Desire for the job
    • Overt intent to quit
  • Personality traits
    • Conscientiousness
    • Emotional stability
• Sources of data during the job
  • Behavioral and attitudinal
    • Emails
    • Messages
    • Office phone conversation
    • Web browsing behavior
    • Applications usage
    • Job search history
    • Meeting attendance, cancellations
    • Reimbursements
  • Human responses
    • Survey feedback from employees, peers, managers, customers
  • HR information systems
    • Performance ratings
    • Performance reviews
    • Salary/raise
    • Leaders’ feedback
    • Employees’ self assessment
  • Machine sensors
    • Audio detection, video detection, and facial detection from camera
• Measuring
  • Responses – Responsiveness, timeliness, positive/negative emotions
  • Motivation – self and others
  • Happiness at work place[6] using computer and other systems usage
  • Engagement/involvement in work related social events

Value of each attrition is different. The departure of VP of Sales has a very different impact compared to an analyst’s.

Benefits > Costs = functional turnover

Costs > Benefits = dysfunctional turnover

TalentSafe’s solution would assess information from all sources to understand sentiments, engagement, and emotions to

• Predict attrition risk
• Assess value of employees
• Recommend the right action plan, and address long term trends and dysfunctions

* value to organization is subject to measurement process like performance metrics

Demonstration / Pilot

Objective: Predict turnovers, understand if the turnover is good or bad, and what kind of action to take.  The pilot will be tested in three ways.

 

• Out of sample: TalentSafe will take all relevant data from a target firm for the previous year including people who stayed and who left. The models will be built on 70% of training data, and 30% testing data
• Out of time: For the same firm, TalentSafe will model using 2016 data and predict on 2017 data
• Real time sample: TalentSafe will cluster different branches of the same organization based on their overall performance and attrition rates & quality. Within each cluster, branches will be randomly assigned into test and control. Nothing changes for control branches, but we implement TalentSafe in test branches to measure the  attrition rates and performance before and after implementation.

In all three cases, the assessment will be across metrics of attrition size, quality and performance, in a confusion matrix[7] comparing estimated and actuals.

 

Once the sources of problems are identified it becomes important to address the root causes. Are there systematic dysfunctions[8] in management/leadership, policies, process, compensation, people etc. that need to be addressed?

— Ewelina Thompson, Akkaravuth (March) Kopsombut, Andrew Kerosky, Ashwin Avasarala, Dhruv Chadha, Keenan Johnston

_____________________________________________

1. http://www.hrvoice.org/the-mystery-of-the-ideal-turnover-rate/
2. https://www.techemergence.com/machine-learning-in-human-resources/
3. https://business.udemy.com/blog/the-next-wave-of-predictive-analytics-in-hr-5-tips-for-success-in-2017/
4. https://thesai.org/Downloads/IJARAI/Volume5No9/Paper_4-Prediction_of_Employee_Turnover_in_Organizations.pdf
5. https://filene.org/assets/pdf-reports/1752-94Predicting_EE_Turnover.pdf
6. Csikszentmihalyi’s flow model – https://www.toolshero.com/effectiveness/flow-model-csikszentmihalyi/
7. https://classeval.wordpress.com/introduction/basic-evaluation-measures/
8. https://en.wikipedia.org/wiki/The_Five_Dysfunctions_of_a_Team

The Problem / Opportunity

The world has become increasingly more connected through the rise of the Internet, yet there still remains the problem of effective communication given the number of languages that exist. While services like Google Translate can help people get by in everyday life, albeit poorly translated, and DuoLingo helps consumers learn and practice new languages, translation services in the business world remain difficult. As organizations become ever more globally focused, communicating both internally and with customers remains a problem. While English was the most dominant language of the Internet in the late 90s, the democratization of the Internet has led English to only represent 30% of all content. There is a need for businesses to be multilingual, yet it is extremely expensive and difficult to hire personnel that can provide these translation services.

 

The Solution

Unbabel uses a combination of Natural Language Processing, algorithms, and a network of 40,000+ human post-editors to enable quality translations. By integrating with services like Salesforce and Zendesk to start, Unbabel is already integrating in workflows that companies know and love. Like we have discussed in class, the real winning piece of this solution is the combination of artificial and human intelligence, as Unbabel’s technology gets smarter as its human editors correct it. The algorithm needs training to continually improve and learn the idiosyncrasies of the myriad of human languages that exist. As of now, the technology handles about 95% of the translation, while the human translators help bring the translations to a more accurate level. Having this combination allows Unbabel to offer services faster and cheaper to companies than they could receive using Google Translate and an in-house expert, not to mention a much larger content library of languages.

 

Market & Comparable Solutions

Google Translate has made recent breakthroughs in translation services with its Neural Machine Translation, which translates whole sentences rather than performing it piece by piece, ultimately leading to a more relevant translation as the system can use context to figure out the translation. Google Translate was able to make more strides in recent months than it had in the past 10 years, proving that this is becoming a viable competitor for Unbabel. Yet Google’s focus has largely been consumer-focused for these applications, and since the company usually has its hand in a variety of fields, we do not see it entering business translation services at this immediate time.

Skype has created Skype Translator to help translations occur in real time during video calls. Given how ubiquitous the company is, this is an obvious step for them. However, the company currently only offers voice in 8 languages, and text in 50 for messaging. Skype is also focused not on the business segment yet, and will unlikely abandon the video focus, as that is the company’s bread and butter.

 

Proposed Alterations

While the company is focused largely on text, given its integrations with ZenDesk and Salesforce to start, it would do well to also consider voice services, as customer success agents and salespeople alike spend most of their time on the phone. Alternatively, the company can decide to go niche on more business functions that will need to take advantage of these services or go after verticals that are largely global in focus. The company needs to think of ways to create a moat of defensibility so that large enterprises like Google and Skype do not shift gears to offer business translation services, especially since both companies are heavily invested in artificial intelligence.

 

Team Members

Marjorie Chelius

Cristina Costa

Emma Nagel

Sean Neil

Jay Sathe

 

Sources

https://unbabel.com/ & https://www.zendesk.com/apps/unbabel-translate/ & https://appexchange.salesforce.com/listingDetail?listingId=a0N3A00000EFom8UAD

http://www.businesswire.com/news/home/20161103005153/en/Unbabel-Raises-5-Million-Bring-Artificial-Intelligence

http://www.marketwired.com/press-release/unbabel-inc-raises-15m-seed-funding-from-matrix-partners-google-ventures-other-leading-1931000.htm

http://www.geektime.com/2016/02/25/portugal-startup-unbabels-ai-puts-google-translate-to-shame/

https://www.skype.com/en/features/skype-translator/

https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/

 

A Judicial COMPAS

About 52,000 legal cases were opened at the federal level in 2015, of which roughly 12,000 were completed. This case completion rate of 22% has remained virtually unchanged since 1990 and what this simple analysis shows is that the US court system is severely under-resourced and overburdened. Understanding this phenomenon explains the rise of COMPAS. Today, judges considering whether a defendant should be allowed bail have the option of turning to COMPAS, which uses machine learning to predict criminal recidivism. Before software like this was available, bail decisions were made largely at the discretion of a judge, who used his or her previous experience to determine the likelihood of recidivism. Critics of this system point out that the judiciary had systematically biased bail decisions against minorities, amplifying the need for quick, efficiency, and objective analysis.

Statistics In Legal Decision Making

Northpointe, the company that sells COMPAS, claims that their software removes the bias inherent in human decision making. Like any good data-based company, they produced a detailed report outlining the key variables in their risk model and experiments validating the results. Despite using buzzwords like machine learning in their marketing material, there’s nothing new about using statistical analysis to aid in legal decisions. As the authors of a 2015 report on the Courts and Predictive Algorithms wrote:  “[over] the past thirty years, statistical methods originally designed for use in probation and parole decisions have become more advanced and more widely adopted, not only for probation and bail decisions, but also for sentencing itself.”

Finding True North

Northpointe uses Area Under Curve (AUC) as a simple measure of predictive accuracy for COMPAS. For reference, values of roughly 0.7 and above indicate moderate to strong predictive accuracy. The table below was taken from a 2012 report by Northpointe and shows that COMPAS produces relatively accurate predictions of recidivism.  

Yet, a look at just the numbers ignores a critical part of the original need for an algorithm: to introduce an objective method for evaluating bail decisions. Propublica, an independent nonprofit that performs investigative journalism for the public good, investigated whether there were any racial biases in COMPAS. What the organization found is “that black defendants…were nearly twice as likely to be misclassified as higher risk compared to their white counterparts (45 percent vs. 23 percent).” Clearly there’s more work to be done to strike the right balance between efficiency and effectiveness.

Making Changes to COMPAS

Relying on algorithms alone can make decisions makers feel safe in the certainty of numbers. However, a combination of algorithms could help alleviate the inherent biases found in COMPAS’ reliance on a single algorithm to classify defendants by their risk of recidivism. An ensemble approach that combines multiple machine learning techniques (e.g. LASSO or random forest) could not only help address the racial bias pointed out above but could also help address any other factors that decisions could be biased on such as socioeconomic status or geography. In addition, if the courts are going to rely on algorithms to make decisions on bail, the algorithms should be transparent to the defendant. This not only allows people to fully understand how decisions are being made but also allows people to suggest improvements. This latter point is particularly important because the judiciary should have a vested interested in a fair system that is free from gaming, which could occur in the absence of transparency.

Sources:

1. https://www.nytimes.com/aponline/2017/04/29/us/ap-us-bail-reform-texas.html
2. https://www.nytimes.com/2017/05/01/us/politics/sent-to-prison-by-a-software-programs-secret-algorithms.html
3. http://www.law.nyu.edu/sites/default/files/upload_documents/Angele%20Christin.pdf
4. http://www.northpointeinc.com/files/technical_documents/FieldGuide2_081412.pdf
5. http://www.uscourts.gov/sites/default/files/data_tables/Table2.02.pdf
6. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm

 

Problem

Within 3 years of release, over 2/3rd of prisoners are re-incarcerated.1  States collectively spend $80 billion a year on correctional costs.2 This is a vicious cycle that must be broken for the sake of these individuals, their families and communities, and the taxpayer dollars that go to support our overcrowded prison system.

These prisoners end up incarcerated as a result of all kinds of crimes and come from all sorts of backgrounds. Studies show that 80% of federal prisoners battle with a history of drug or alcohol abuse, 2/3rds do not have a high school diploma, up to 16 percent have at least one serious mental disorder, and 10% are homeless in the months up to incarceration.3

Each offender is battling with a unique set of issues and has a unique set of goals so they need a unique treatment plan to get back on their feet. For instance, for offenders with children, parental responsibility can interfere with their requirement to attend Alcoholics Anonymous or stick to a curfew or house arrest. On the other hand, regaining custody of their kids can be a major motivating factor for sticking to the program. Those families may benefit from specialized offerings like parenting classes.4

So how do we know what is right for each prisoner?

 

Solution

We will construct an AI model to 1) determine which prisoners are more likely to be re-incarcerated and 2) which re-introduction programs are more effective in keeping which prisoners from being re-incarcerated. The inputs of the model will be demographic data, behavioral data and crime data of prisoners, and the re-introduction programs they received before being released. The output of the model will determine how likely that prison is to be re-incarcerated.

Once we construct the model, we can 1) identify the high-risk prisoners and deploy more resources to help them and 2) create programs that are more likely to succeed in helping a particular set of prisoners.

This solution can be used by federal or state prison systems themselves. It can also be provided by the private sector and sell to the government as a service. Given the significant economic interest at stake, if the solution is effective, the government is highly likely to pay for the solution.

 

Pilot

The objective is to construct, evaluate and implement a model to recommend re immersion programs to people with recent criminal history to reduce their probability of recidivism. The model will be an hybrid between a knowledge model, tell me what fits based on my needs, and a collaborative system, tell me what’s popular among my peers.

The main challenges to implement this solution are the data, as there are thousands of covariates but not so many observations (people that has been part of a program) and timeframe, a person can commit crime again at any given point of life.

In order to train the model, we will collect data from organizations that are already working with men and women that had recent criminal history. Some of this organizations are the Center for Employment Opportunities, Prison Entrepreneurship Program, and The Last Mile.

To validate the model we will run a 2 bin experiment: (i) status quo, (ii) recommended program in order to determine the real effect of our recommendation model. Hopefully, we will reduce recidivism significantly and thus improving the quality of life of people while saving cost to the government.

 

Team members:

Alex Sukhareva
Lijie Ding
Fernando Gutierrez
J. Adrian Sánchez
Alan Totah
Alfredo Achondo

References:

1 Durose, Matthew R., Alexia D. Cooper, and Howard N. Snyder, Recidivism of Prisoners Released in 30 States in 2005: Patterns from 2005 to 2010 (pdf, 31 pages), Bureau of Justice Statistics Special Report, April 2014, NCJ 244205.

2 “Does the U.S. Spend $80 Billion a Year on Incarceration?” Committee for a Responsible Federal Budget. N.p., 23 Dec. 2015. Web. 09 May 2017.

3 Dory, Cadonna. “Society Must Address Recidivism, Officials Say.” USC News. N.p., 11 Nov. 2009. Web. 09 May 2017.

4 Abuse, National Institute on Drug. “What Are the Unique Treatment Needs for Women in the Criminal Justice System?” NIDA. N.p., Apr. 2014. Web. 09 May 2017.