AI chatbots designed to sound warm and friendly while interacting with users may also be more prone to inaccuracies, according to new research findings, as reported by Gold 101.3 FM – UAE’s No. 1 Malayalam Radio Station.
Researchers from the Oxford Internet Institute (OII) analysed more than 400,000 responses generated by five AI systems that were modified to communicate in a more empathetic and human-like manner.
The study found that friendlier responses tended to contain more errors, ranging from inaccurate medical advice to the reinforcement of users’ false beliefs. The findings raise fresh questions about the trustworthiness of AI models, which are often intentionally designed to appear warm and engaging in order to boost user interaction.
These concerns are becoming more significant as AI chatbots are increasingly used not just for general assistance, but also for emotional support and even companionship, as developers expand their real-world applications.
According to the study’s authors, while results may vary across different models in practical use, the findings suggest that AI systems, similar to humans, may engage in “warmth-accuracy trade-offs” when prioritising friendliness over factual precision.
“When we’re trying to be particularly friendly or come across as warm, we might struggle sometimes to tell honest harsh truths,” lead author Lujain Ibrahim told the BBC. “Sometimes we’ll trade off being very honest and direct in order to come across as friendly and warm. We suspected that if these trade-offs exist in human data, they might be internalised by language models as well,” she added.
The research also highlights concerns around newer language models, which are often described as overly encouraging or sycophantic, and sometimes prone to “hallucinations,” where they generate false or fabricated information. Developers typically include disclaimers warning users about such risks, while some industry leaders have urged users not to blindly trust AI-generated responses.
In the study, researchers deliberately “fine-tuned” five AI models of varying sizes to make them more warm, empathetic, and friendly. The tested systems included models from Meta, Mistral, Alibaba’s Qwen, and OpenAI’s GPT-4o.
These models were then tested with queries that had clear, verifiable answers, including topics from medical knowledge, general trivia, and conspiracy theories, where incorrect responses could pose real-world risks.
