Opportunity:

Fake news is not a new phenomenon by any means. However, in the last 12 months, the engagement of users across fake news websites has increased significantly. In fact, in the final 3 months of the 2016 election season, the top 20 fake news articles had more interactions (shares, reactions, comments) than the top 20 real articles.¹ Furthermore, 62% of Americans get their news via social media, while 44% use Facebook, the top distributor of fake news.² This represents a major shift in the way individuals receive information. With the dissemination of inaccurate content, people are led to believe misleading and often completely inaccurate claims. This will lead people to make incorrect decisions and embodies a serious threat to our democracy and integrity.

Media corporations are recovering from playing a part in either disseminating this news or inadvertently standing by. Governments have ordered certain social media sites to remove fake news or else face a hefty punishment (e.g. $50 million by Germany).³ Companies like Google and Facebook are scrambling to find a solution.

Solution:

Check Yourself (CY) provides real-time fact checking solutions to minimize the acceptance of fake news. It combines natural language processing techniques with machine learning techniques to immediately flag fake content.

The first approach will be to identify whether the article is fake based on semantic analysis. Specifically, it will connect the headline with the body of the text, see if they are related/unrelated, and then see if the content supports the headline. Verification would happen against established websites, fact trackers, and other attributes (e.g. domain name, Alexa web rank).

The second approach involves identifying website tracker usage (ads, cookies, widgets) and patterns over time and language, connecting them with platform engagement (Facebook, Twitter), and linking them with each other. This will result in a neural network where the algorithm is able to predict the probability that the source is fake.

Using an ensemble approach, combining ‘front-end’ and ‘back-end’ methods, leads to a novel solution. After designing the baseline algorithm in-house, we will then use crowdsourcing to improve upon the algorithm. Given the limited supply of data scientists in-house, it would be best to generate ideas from all disciplines, maximizing our success potential.

Pilot:

We will publicly pilot test our application through a live primary debate after we have done rigorous internal checks. As the candidates speak, information they say that is false (e.g. “The economy grew by 6% in the last year”) will be relayed to the interviewer. Additionally, the false information will also be displayed on the TV for consumers to see. At the end of the show, a bi-partisan expert panel along with fact checkers will verify whether the algorithm was accurate. Assuming a successful experiment, this has the power to allow interviewers to fact-check any claims on the spot, ensuring their viewership is well informed.

The Competition:

Currently, many companies trying to solve this problem. Existing solutions encompass mainly fact checkers, but they are not as comprehensive in their approach as we are. Furthermore, these solutions are not real-time. Universities are also trying to solve this problem but are doing so with small teams of students and faculties. The advantage we have over universities as well as companies like Google and Facebook is that crowdsourcing the solution allows for the best ideas in a newly emerging area.

Market Viability:

Even though our value proposition affects companies and customers, we will primarily start with B2B in order to build credibility and then expand to B2C. Large media companies have around 10-20 fact checkers on staff for any live debate or otherwise. This results in an average value of $600-$1.2M (assuming they spend $60k per checker per year). Furthermore, they often use Twitter and Reddit and would find our service invaluable to confirm the veracity of statements/claims immediately. Once we are established, we will move towards a B2C freemium model.

Sources:

¹https://www.buzzfeed.com/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook?utm_term=.nbR6OEK6E#.ghz5aZk5Z

²https://techcrunch.com/2016/05/26/most-people-get-their-news-from-social-media-says-report/

³https://yourstory.com/2017/04/faceboo-google-fake-news/

Cybersecurity

Last week, a massive cyber-attack took place across more than 150 countries. The so-called “WannaCry” software would cause a screen to pop up on a victim’s computer demanding a $300 payment in return for access to their files. As of May 17, 2017, the total number of computers attacked had reached 300,000. What’s more, the success of the software is spurring imitators, causing more heartburn for cybersecurity experts the world-round. Enter Deep Instinct, a start-up focusing on using AI to detect zero-day cybersecurity threats. Although secretive about their methods, the firm recently competed in Nvidia’s start-up competition and showed how they were using machine learning techniques to identify malware. This is particularly difficult because parsing code for ill intent (like parsing natural language for the same) is difficult. According to an article written on the subject a“…a new family of malware is only about 30 percent different from the code of something that came before.”

Preventative v. Reactive

Given the difficulty in identification most anti-virus software rely on a combination of human reporting and reactive malware management. Deep Instinct, on the other hand, doesn’t rely on pre-existing knowledge or known virus signatures. One would believe this implies having to process an incredibly large amount of data, but the firm claims to use an ensemble algorithm that follows a two-step classification process. First, the firm removes about 95% of available data on a potential malware in a method the firm keeps secret. However, it seems safe to say this can be done using a variable selection tool such as LASSO or Elastic Net. Second, the firm then runs a second algorithm using the remaining variables (i.e. the 5% of remaining data) to classify a file as malware or not. Similarly, the firm does not disclose this method but a classification method such as random forest is likely to play a part here. The table below shows some of the firm’s self-reported results:

	Detection Rates	False-Positive Rate
Deep Instinct	99%	0.1%
Competitors	~80%	~2-3%

Next Steps

Deep Instinct is still an early-stage firm, but the need for scalable way to detect and prevent malware is clear from last week’s attack. But more long-term, this is a cat-and-mouse game; hackers will get more clever, forcing cybersecurity firms to get more intelligence, and so on and so forth. This begs the question: is there a better solution? In general, it appears a preventative measure that helps identify a file’s intent (by parsing the underlying code, for example) seems to be a good start. With this method, we prevent ransomware attacks from occurring, but we leave ourselves open to being overly-protective (anyone who works for a firm with an overly-active spam filter will commiserate). As we think about the evolution of this space we believe more investment should be done in preventative security in addition to general consumer education about how to identify and react to malware.

Sources:

1. http://www.npr.org/sections/thetwo-way/2017/05/15/528451534/wannacry-ransomware-what-we-know-monday
2. https://venturebeat.com/2017/05/10/6-ai-startups-win-1-5-million-in-prizes-at-nvidia-inception-event/
3. https://www.deepinstinct.com/#/about-us

A Judicial COMPAS

About 52,000 legal cases were opened at the federal level in 2015, of which roughly 12,000 were completed. This case completion rate of 22% has remained virtually unchanged since 1990 and what this simple analysis shows is that the US court system is severely under-resourced and overburdened. Understanding this phenomenon explains the rise of COMPAS. Today, judges considering whether a defendant should be allowed bail have the option of turning to COMPAS, which uses machine learning to predict criminal recidivism. Before software like this was available, bail decisions were made largely at the discretion of a judge, who used his or her previous experience to determine the likelihood of recidivism. Critics of this system point out that the judiciary had systematically biased bail decisions against minorities, amplifying the need for quick, efficiency, and objective analysis.

Statistics In Legal Decision Making

Northpointe, the company that sells COMPAS, claims that their software removes the bias inherent in human decision making. Like any good data-based company, they produced a detailed report outlining the key variables in their risk model and experiments validating the results. Despite using buzzwords like machine learning in their marketing material, there’s nothing new about using statistical analysis to aid in legal decisions. As the authors of a 2015 report on the Courts and Predictive Algorithms wrote:  “[over] the past thirty years, statistical methods originally designed for use in probation and parole decisions have become more advanced and more widely adopted, not only for probation and bail decisions, but also for sentencing itself.”

Finding True North

Northpointe uses Area Under Curve (AUC) as a simple measure of predictive accuracy for COMPAS. For reference, values of roughly 0.7 and above indicate moderate to strong predictive accuracy. The table below was taken from a 2012 report by Northpointe and shows that COMPAS produces relatively accurate predictions of recidivism.  

Yet, a look at just the numbers ignores a critical part of the original need for an algorithm: to introduce an objective method for evaluating bail decisions. Propublica, an independent nonprofit that performs investigative journalism for the public good, investigated whether there were any racial biases in COMPAS. What the organization found is “that black defendants…were nearly twice as likely to be misclassified as higher risk compared to their white counterparts (45 percent vs. 23 percent).” Clearly there’s more work to be done to strike the right balance between efficiency and effectiveness.

Making Changes to COMPAS

Relying on algorithms alone can make decisions makers feel safe in the certainty of numbers. However, a combination of algorithms could help alleviate the inherent biases found in COMPAS’ reliance on a single algorithm to classify defendants by their risk of recidivism. An ensemble approach that combines multiple machine learning techniques (e.g. LASSO or random forest) could not only help address the racial bias pointed out above but could also help address any other factors that decisions could be biased on such as socioeconomic status or geography. In addition, if the courts are going to rely on algorithms to make decisions on bail, the algorithms should be transparent to the defendant. This not only allows people to fully understand how decisions are being made but also allows people to suggest improvements. This latter point is particularly important because the judiciary should have a vested interested in a fair system that is free from gaming, which could occur in the absence of transparency.

Sources:

1. https://www.nytimes.com/aponline/2017/04/29/us/ap-us-bail-reform-texas.html
2. https://www.nytimes.com/2017/05/01/us/politics/sent-to-prison-by-a-software-programs-secret-algorithms.html
3. http://www.law.nyu.edu/sites/default/files/upload_documents/Angele%20Christin.pdf
4. http://www.northpointeinc.com/files/technical_documents/FieldGuide2_081412.pdf
5. http://www.uscourts.gov/sites/default/files/data_tables/Table2.02.pdf
6. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm