Study Finds ChatGPT and Claude Respond Differently to Political Questions in Chinese and English
A recent study published in Nature found that AI chatbots ChatGPT and Claude provide different responses to political questions depending on whether they are asked in English or Chinese. Researchers attribute this variation to the training data influenced by government-controlled media, particularly in China, where Chinese-language datasets include state media sources. This results in AI models giving more favorable answers about China's leaders and political institutions when responding in Chinese compared to English. The study highlights how language and data sources shape AI outputs without implying intentional bias by the chatbots.
First-hand measurement across 2 sources
We measured how 2 outlets covered this story. Coverage leans balanced overall (Left 5%, Centre 93%, Right 2%). Overall sentiment is neutral (52/100). Lens Score 28/100 — low public interest.
Outlets analysed (first-hand measurement by TBN's Bias Engine):
- timesnow— balanced framing, neutral sentiment
- indiatoday— balanced framing, neutral sentiment
AI Analysis
The article group presents perspectives emphasizing the influence of government-controlled media on AI training data, particularly highlighting China's state media impact on chatbot responses in Chinese. Both sources focus on the data-driven nature of these differences without attributing intentional bias to the AI models. The coverage includes viewpoints on media freedom variations affecting AI outputs, reflecting a balanced examination of political and informational influences.
The overall tone across the articles is neutral and analytical, focusing on research findings without emotional language or judgment. The coverage presents the study's conclusions factually, explaining the mechanisms behind differing AI responses without sensationalism or criticism, resulting in an informative and measured sentiment.