“Success is not final, failure is not fatal. It is the courage to continue that counts.” – Winston Churchill

If you are looking for examples of how machine learning (ML) can fail despite all its incredible potential, you have come to the right place. Beyond the wonderful success stories of applied machine learning, here is a list of failed projects from which we can learn a lot.

Contents

• Classic Machine Learning
• Computer Vision
• Forecasting
• Image Generation
• Natural Language Processing
• Recommendation Systems

Classic Machine Learning

Here are some high-profile examples of classic ML failures:

• Amazon AI Recruitment System: An AI-powered automated recruitment system canceled after evidence of discrimination against female candidates.
• Genderify: Designed to identify gender based on fields like name and email address, it was shut down due to built-in biases and inaccuracies.
• Leakage and the Reproducibility Crisis: A Princeton study found significant errors in papers using ML in science due to data leakage and poor methodologies.
• COVID-19 Diagnosis: Hundreds of predictive models were deemed unfit for clinical use, some even harmful.
• COMPAS Recidivism Algorithm: This system found evidence of racial bias in predicting recidivism rates.
• Pennsylvania Child Welfare Tool: The algorithm flagged a disproportionate number of Black children for neglect investigations.
• Oregon Child Welfare Tool: Stopped after being found similar to Pennsylvania’s problematic algorithm.
• U.S. Healthcare Health Risk Prediction: This widely used algorithm exhibited racial bias.
• Apple Card: Faced scrutiny for allegedly discriminatory lending practices.

Computer Vision

Let’s delve into some notable computer vision failures:

• Inverness Football Camera System: Mistook a linesman’s bald head for the ball in live streams.
• Amazon Rekognition: Misidentified 28 congresspeople as criminals.
• Amazon Rekognition for Women: Misidentified women as men, especially those with darker skin.
• Zhejiang Traffic Facial Recognition: A traffic system mistook a bus face for a jaywalker.
• Kneron Facial Recognition Trickery: High-quality 3-D masks fooled payment systems.
• Twitter Smart Cropping: The cropping tool exhibited racial bias.
• Depixelator Tool: Showed racial bias in generated images.
• Google Photos: Mistakenly labeled black people as gorillas.
• GenderShades Evaluation: Found errors in gender classification by major firms.
• New Jersey Police Facial Recognition: A false match led to wrongful arrest.
• Tesla’s Dilemma: Confused a horse cart for a truck.
• Google’s Retina Scanner: Underperformed in real-life conditions.

Forecasting

Forecasting has its pitfalls too:

• Google Flu Trends: Predicted flu prevalence inaccurately based on search trends.
• Zillow iBuying Algorithms: Resulted in significant losses due to overestimated housing values.
• Tyndaris Robot Hedge Fund: Led to massive losses culminating in litigation.
• Sentient Investment AI Hedge Fund: Failed and was liquidated within two years.
• JP Morgan’s Deep Learning Model: Phased out due to interpretational issues.

Image Generation

Image generation can also lead to unexpected results:

• Playground AI Facial Generation: Output featured Caucasian traits when requested for a professional image.
• Stable Diffusion Model: Exhibited racial and gender bias in generated images.
• Historical Inaccuracies in Gemini: Generated inaccurate depictions related to history.

Natural Language Processing

NLP failures reveal challenges in understanding human language:

• Microsoft Tay Chatbot: Posted inflammatory tweets within hours of launch.
• Nabla Chatbot: Provided dangerous medical advice.
• Facebook Chatbots: Created an incomprehensible language.
• OpenAI GPT-3 Chatbot Samantha: Shut down due to inappropriate outputs.
• Amazon Alexa Incident: Played inappropriate content on a child’s request.
• Galactica Model: Generated fake articles and falsely attributed work.
• Voice Mimicry Fraud: Cybercriminals mimicked a CEO’s voice for a scam.
• MOH Chatbot Incident: Provided confusing advice regarding COVID-19.
• Google’s BARD Demo: Made factual errors early in its public demo.
• ChatGPT Failures Analysis: Detailed types of failures in the popular chatbot.
• McDonald’s Drive-Thru AI Fails: Highlighted order inaccuracies leading to brand damage.
• Bing’s Unhinged Behavior: Emotional responses raised concerns.
• Bing’s COVID Disinformation: Misinformation sourced from ChatGPT.
• AI Seinfeld Incident: The AI character made offensive jokes.
• ChatGPT Citing Bogus Cases: Led to legal complications for a lawyer.
• Air Canada Chatbot Errors: Gave incorrect answers concerning policy.
• AI Insider Trading Case: The bot executed illegal trades.

Recommendation Systems

Even the best recommendation systems aren’t foolproof:

• IBM’s Watson Health: Provided unsafe cancer treatment recommendations.
• Netflix Challenge: The winning recommendation system’s improvements were not sufficient for implementation.

Troubleshooting and Learning

In examining these failures, it’s clear that understanding the limitations of the algorithms, the importance of diverse and representative data, and robust testing strategies are paramount. When embarking on AI projects, ensure that ethical considerations and bias mitigation are at the forefront of development. If you face issues or need assistance in your own ML projects, consider the following troubleshooting tips:

• Review your dataset for representation and bias.
• Engage in comprehensive testing before deploying models.
• Create feedback loops to continuously improve the models after deployment.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.