From $1 Tahoes to Lawsuits: Adopting AI without safeguarding it

AI has immense potential. Referring to Wardley Map, it's a product for individual consumers and it's custom built via MCP servers and RAG for institutional consumers as I currently see it.

‍

Where should organisations leverage AI to maximise efficiency and effectiveness?

Clarifying the problem that needs to be solved instead of thinking where AI could be applied in the first place. AI is an accelerator. It can boost time-to-market, but financial or reputational disasters if not applied strategically.

Implementing AI just because of FOMO or with a hope to spearhead the next revolution has backfired for organisations such as Duolingo, MSN, CNET, and more. Therefore, adding AI is not the solution per se. That's where root cause analysis help identify which problems to solve. Then the next question is: "What could help us solve that problem?".

‍

Using Root Cause Analysis to Pinpoint Problems Worth Solving

The larger an organisation grows, the harder it gets to see the big picture. Departments often apply local fixes that put a band-aid on symptoms, leaving much bigger learning and improvement opportunities on the table. Rather than stopping at identifying topical pain points, root cause analysis helps organisations understand why problems occur. Several techniques assist in this process:

Fishbone (Ishikawa) Diagram

Fishbone diagram categorises potential causes into factors such as people, processes, equipment, or external influences. It involves starting with the problem, and walking backwards to identify the root causes.

5 Whys Analysis

Five whys is a root-cause analysis technique that drills down into underlying issues by repeatedly asking “why” until the root cause is uncovered.

Process Simulations

Process simulation tests potential impact of changes in a virtual environment before implementation. It is a quantitative technique, therefore, requires reliable operational data so you can closely replicate your organisation's current-state which is also known as digital twin.

There are tools out there such as SAP Signavio and Celonis that enable process mining, modelling, and simulation.

Without understanding the root causes of organisational challenges and customer needs, AI accelerates loss. If you asked 5 Whys to Duolingo going full AI, the answer wouldn't be that the customers want to talk to robots. Or if you take MSNs AI-content creation through the fishbone diagram, you won't end up with "because the readers want fake news and bland content".

Therefore, it is more important to answer the following questions.

• What problems are worth solving?
• What tools (be it AI or not) could help us solve them.

Implications of Applying AI without Guardrails

Other than subpar user experience, there have also been more direct cost implications of applying AI without sufficient guardrails.

• A Chevy dealership deployed an AI chatbot that was quickly "jailbroken" by users, who convinced it to agree to sell a 2024 Chevy Tahoe for $1.
• United Healthcare had to cop a lawsuit due to their AI model denying healthcare claims from elderly Medicare Advantage patients despite the physician recommendations. Their LLM was measurably flawed. It had 90% error rate.
• A Taco Bell customer has crashed the system by ordering 18000 water cups.

There are lots of examples to applying AI without sufficient guardrails, and there will be a lot more.

Final Thoughts

You might invest millions in new tech when the root cause was a $50 training manual that nobody could understand. It's crucial to understand where technology fits in and solved real business problems to avoid misdirected investments.

Which part of your process is currently a 'black box' where AI might actually do more harm than good?

‍