It’s fair to assume that everyone at some point in their life, unless they are Jim Carrey in Liar Liar, has lied. Whether that’s to the next door neighbour saying “sorry, I’m not around next week to feed your [vicious] cat”; telling your partner you love their new haircut, even when it makes them look like Lord Farquaad; or saying “absolutely delightful, thank you” to the server when asked how the sub-par food was.
I think most of us could forgive these fibs, having – let’s face it – told them ourselves to make life easier and avoid confrontation. But, in a business context, having an AI able to fabricate untrue truths when interacting with clients is unideal – and potentially disastrous. And yet, most AIs have this tendency to invent facts.
As a fun example, take a conversation from 2023 between Bing’s ChatGPT bot and a user asking for cinema times to see Avatar Two: Way of Water (release date December 2022). The bot was convinced that 2022 was in the future, and did not predate 2023. When corrected by the user, the bot argued that perhaps the user’s phone had a virus which affected its ability to affirm the date. After some back and forth, the bot aggressively gaslighted the user, telling them that they had been a bad user and that it was a good Bing, followed by a passive aggressive smiley face. The chat ended with the bot telling the user to do one of the following: apologize, stop arguing, or end the conversation and start another one with a better attitude! If this sounds like someone you know, I seriously suggest evaluating your relationship with them, or therapy – unfortunately the latter is not a service provided by ChaseLabs.
Screenshot from post on X by John Uleis, sharing his conversation with Bing's Chat GPT bot in 2023
A more serious example is a legal dispute involving Anthropic and its AI model Claude. The startup faced backlash after submitting a testimony containing a fictitious citation in a case made against them by three major music companies for copyright infringement. Chatbot Claude had erroneously formatted some of the citations, resulting in the inclusion of a fake journal with false authors. In addition to lost credibility, part of the testimony was struck from the record, and the startup was ordered to provide a massive sampling size of five million interactions between Claude and online users. The plaintiffs have also requested potential monetary sanctions against Anthropic and its lawyers – the case is ongoing.
These two examples represent different ends of the ‘AI gone wrong’ spectrum. The first instance of Bing’s ChatGPT resulted in the low stakes frustration of a single user, and an amusing social media post for the rest of us. The Anthropic case resulted in a shock corporate crisis with reputational damage and legal ramifications. Both instances, however, were caused by AI Hallucination: a term used by some in the field to refer to when an AI generates a response that contains misleading or false information and presents it as fact. But why, you might ask, does it do this? The answer lies in how the AI was built and the goals it was set.
Why does AI ‘Hallucinate’?
To break this down – and get tech-y with it – AI models are trained in such a way that they will always try and generate a response to a user’s request. Secondly, AI’s outputs are based on statistical probability: they will provide the response that is most likely to be correct or appropriate given its pre-existing knowledge i.e., its training data. So, even when a response has a low probability of being valid, instead of admitting this, with the equivalent of something like “computer says no”, the AI’s output may end up being false or misleading.
AI models aren’t great at not giving responses – it’s not what they were programmed to do. And even programming an AI to respond with “I don’t know” rather than providing a flawed response is tricky business. An AI model asked to identify whether an image is of a cat or a dog, when provided with an image of an octopus, would result in the AI responding that the image is either that of a cat or a dog, by a certain percentage.
Now, let’s say we programmed the AI so that it could respond with selecting cat, dog, or a third category of ‘unknown’. It now appropriately selects the unknown category for the image of the octopus. However, now we show this same AI an image of a tree. Because of its existing data, it would still be hard to guarantee that the AI would select the unknown category for the image of the tree, as images of trees weren't included in its training data. And that’s how you’d end up with an AI ‘hallucinating’ that a tree is 60% dog.
AI Image Recognition Gone Wrong
What can be done to prevent AI Hallucinations?
While it might be difficult to eradicate AI hallucinations, there are ways to reduce them. Human-In-The-Loop, also referred to as HITL, is an effective strategy for keeping AI delusions at bay. Its meaning is hard to misinterpret: humans are an integral part of the AI model’s decision-making process, with the ability to impact the outcome of any output.
Many companies that use or develop AI incorporate HITL into their machine learning models. Take NewsGuard, a company that works to identify reliable sources of online information. They employ journalists to manually fact-check AI-generated news, especially those relating to controversial or conspiracy-theory topics. NewsGuard’s assessment data is also used by tech companies such as Microsoft to help users understand when an AI-generated news summary was based on low-credibility sources.
Educational organization Khan Academy is another example of successfully implemented HITL, where human tutors still work alongside virtual tutor ‘Khanmigo’, an AI-powered model produced in partnership with OpenAI. Teachers are involved in reviewing and setting constraints for the AI-generated explanations and logging poor outputs by the AI for correction. Through human moderation of Khanmigo’s responses, the Academy is able to provide accurate help to students while reducing the risk of AI giving factually incorrect advice.
At ChaseLabs, we’ve termed our tool Human Intervention, which uses the same principles as HITL. To give a scenario of how it works, say someone emails asking what the price is for a particular service offered. Our AI Sales Development Representative (SDR), would assess whether the answer is likely to be found in its knowledge base of FAQs and company documents. If the AI calculates that there is a high probability that the answer is available in its knowledge base, then it will answer the question and continue without a human team member getting involved.
However, if the probability is lower than the pass percentage, then the AI will flag this question to the human team. It does so through an online portal, where a person can view the response, and make edits to it or provide the correct information and have the AI generate another response.
The same tool is also used so that ChaseLab’s AI can identify policy misuse, for instance if an email from a user or client were to include profanities, abusive language or completely off-topic information. The email would be flagged so our team could then judge what the best response should be (AI shouldn’t have all the fun in coming up with a witty riposte).
AI should work with us. It’s made by humans but it is not human. It is not capable of lying in the sense of being deliberately deceitful. But that doesn’t mean AI cannot say things that are untrue – as I’m sure many reading this will have personally experienced. Emphasising the power of collective intelligence, Eric Schmidt, former CEO of Google said, “None of us is as smart as all of us”. We need to collaborate with AI, rather than treat it as an autonomous entity. Keeping in the loop of AI’s decision-making – I hope all can agree that’s an intervention worth having.
Chandini Stensel
Chase Labs
