Can you count on AI to give you a good summary of the news? A recent investigation by the BBC raises major questions. More and more people are using chatbots to get rapid summaries, thus accuracy is more critical than ever. The BBC put ChatGPT, Gemini, Copilot, and Perplexity to the test with 100 genuine news pieces to see how effectively AI tools can do real journalism.
The results were shocking and very worrying. These tools typically failed in important ways, such as giving false information, making up quotes, giving bad health advice, and changing the tone of the conversation. This study isn't only about mistakes; it's also about how quickly false information may spread when people trust AI without checking the facts.
The BBC Put Leading AI Chatbots to the Test
The BBC chose to check the accuracy of popular AI systems that summarize news. They chose four popular chatbots: ChatGPT, Google Gemini, Microsoft Copilot, and Perplexity AI. Each tool was given 100 real news stories from the BBC's website. The articles covered various topics such as health, politics, and global affairs. The job was to either summarize the articles or answer questions about them.
The goal of this project was to find out how trustworthy and accurate these AI technologies are when it comes to doing real journalism. It wasn't only to check speed or grammar; it was also to see how effectively the tools kept truth, tone, and context. It was the first big research of its kind by a well-known international news outlet.
Half the Summaries Failed to Get the Basics Right
The BBC's analysis found that more than half of the summaries made by AI were not correct. More than half of the responses had significant issues, like missing information, changed meanings, and bad context. These weren't just small blunders; they altered the way the news was reported significantly. Some important facts were left out, or the order of events was mixed up.
Some descriptions made complicated subjects too simple, while others completely missed the point. It means that people who merely read AI summaries would not grasp the whole narrative. These high error rates are alarming and highlight the risk of using automation in journalism when so many people only read summaries instead of complete pieces.
AI Gets the Facts Wrong — and That's a Problem
The BBC found that 19% of AI-generated answers included factual inaccuracies, which made them quite worried about their reliability. It wasn't just mistakes; AI programs also gave incorrect names, dates, and statistics. In several cases, quotes were wrongly given to the wrong person. In some cases, the data was manipulated to make it appear better than it really was or was made up entirely.
These mistakes might easily lead readers astray because many people trust AI to provide them with speedy and accurate answers. When talking about politics, science, or public safety, it's extremely bad to get facts wrong. Even a tiny mistake might change how someone thinks or acts. These results reveal that AI tools aren't always accurate or true, even when they sound sure of themselves.
Invented Quotes Are a Serious Red Flag
One of the most worrying things we found was that 13% of the summaries had quotes that had been altered or fabricated. AI tools either modified what someone said or made up new quotes from scratch. Quotes are essential in journalism. They reflect exactly what was said and how it was said. When AI changes them, it not only gives a false picture of the person, but it also puts bogus stories at risk of spreading.
The BBC analysis uncovered cases where famous people seemed to say things they never said. It could hurt people's reputations or start unnecessary conflict. Making up quotes is a severe journalistic crime, and the fact that AI technologies do this without warning shows how risky it is to use them without human supervision. That's a good reason to examine the facts before posting anything made by AI.
When AI Misreads the Mood of the News
The BBC also found that AI algorithms typically modified the emotional tone of major news pieces. In other words, they may make a political crisis look like a light argument or a health catastrophe look like a casual update. When something is misrepresented, it changes how readers perceive the importance or urgency of a problem.
The tone of a news item has a big effect on how people react. When AI makes things less serious or adds a tone that is too upbeat or unclear, it makes it hard for the reader to understand the underlying meaning. It is especially risky when it comes to breaking news or sensitive issues. The AI doesn't know how to read between the lines or how heavy some sentences are.
Misleading Health Advice Could Have Real-World Impact
The BBC investigation gave a concrete example of how AI can give out false information that could be harmful. Google Gemini said that the NHS doesn't want people to vape, which is not true at all. The NHS advocates vaping as a safer way for people to quit smoking. This mistake isn't only wrong; it can also change how people make health decisions. If a smoker read that summary, they might not use a beneficial item because they thought it was wrong.
This example indicates that AI is not precise or careful enough for health reporting. The summary from the chatbot sounded sure, yet it was inaccurate. This one mistake shows a bigger problem: AI can change even official health advice, which can be harmful. That's why it's quite dangerous to rely on unconfirmed summaries for medical or safety information.
Conclusion:
The BBC's investigation makes it obvious that AI techniques are not yet good enough to summarize news. These summaries can easily distort the truth by introducing errors, fake quotes, and misleading tones. When people rely on these tools for quick updates, false information can spread quickly, which is especially dangerous in health, politics, and crisis reporting. AI has a lot of promise, but it still needs a lot of human monitoring. Users should check the facts and not trust AI-generated material without verifying it.
