Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews
#Misinformation#Chinese web#Search engines#Large Language Models#AI overviews#Fact-checking#Information retrieval#Digital ecosystem
📌 Key Takeaways
Researchers evaluated factual accuracy of search engines, LLMs, and AI overviews in Chinese web ecosystem
Study used dataset of 12,161 real Chinese Yes/No questions from search logs
Significant differences in accuracy and topic variability found across systems
Research estimates potential misinformation exposure for Chinese users across regions
Findings highlight structural risks in AI-mediated search and need for more reliable tools
📖 Full Retelling
Researchers led by Geng Liu from multiple institutions have published a comprehensive study on December 15, 2025, evaluating misinformation exposure in the Chinese web ecosystem by comparing traditional search engines, standalone Large Language Models, and AI-generated overview modules, addressing growing concerns about factual accuracy as AI systems increasingly provide direct answers to user queries. The study introduces a fact-checking dataset of 12,161 Chinese Yes/No questions derived from real-world online search logs, providing researchers with a robust foundation for evaluating the performance of different information systems. The researchers developed a unified evaluation pipeline to systematically compare how these three paradigms handle factual queries, revealing substantial differences in accuracy and topic-level variability across systems. By combining their performance analysis with real-world Baidu Index statistics, the team was able to estimate potential exposure to incorrect factual information for Chinese users across different regions in China. The findings highlight significant structural risks in AI-mediated search, particularly as these systems become more integrated into daily information access, underscoring the critical need for more reliable and transparent information-access tools in the digital world, especially for non-English speaking populations.
🏷️ Themes
Information reliability, AI technology, Digital media
Misinformation is incorrect or misleading information. Whereas misinformation can exist with or without specific malicious intent, disinformation is deliberately deceptive and intentionally propagated. Misinformation is typically spread unintentionally, mostly caused by a lack of knowledge, an error...
Search engines, including web search engines, selection-based search engines, metasearch engines, desktop search tools, and web portals and vertical market websites have a search facility for online databases.
AI Overviews is an artificial intelligence (AI) feature integrated into Google Search that produces AI-generated summaries of search results. The feature has been criticized for its accuracy and for reducing traffic to content websites.
A large language model (LLM) is a language model trained with self-supervised machine learning on a vast amount of text, designed for natural language processing tasks, especially language generation. The largest and most capable LLMs are generative pre-trained transformers (GPTs) that provide the c...
--> Computer Science > Information Retrieval arXiv:2602.22221 [Submitted on 15 Dec 2025] Title: Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews Authors: Geng Liu , Junjie Mu , Li Feng , Mengxiao Zhu , Francesco Pierri View a PDF of the paper titled Misinformation Exposure in the Chinese Web: A Cross-System Evaluation of Search Engines, LLMs, and AI Overviews, by Geng Liu and 4 other authors View PDF HTML Abstract: Large Language Models are increasingly integrated into search services, providing direct answers that can reduce users' reliance on traditional result pages. Yet their factual reliability in non-English web ecosystems remains poorly understood, particularly when answering real user queries. We introduce a fact-checking dataset of 12~161 Chinese Yes/No questions derived from real-world online search logs and develop a unified evaluation pipeline to compare three information-access paradigms: traditional search engines, standalone LLMs, and AI-generated overview modules. Our analysis reveals substantial differences in factual accuracy and topic-level variability across systems. By combining this performance with real-world Baidu Index statistics, we further estimate potential exposure to incorrect factual information of Chinese users across regions. These findings highlight structural risks in AI-mediated search and underscore the need for more reliable and transparent information-access tools for the digital world. Subjects: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY) Cite as: arXiv:2602.22221 [cs.IR] (or arXiv:2602.22221v1 [cs.IR] for this version) https://doi.org/10.48550/arXiv.2602.22221 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Geng Liu [ view email ] [v1] Mon, 15 Dec 2025 15:36:13 UTC (2,078 KB) Full-text links: Access Paper: View a PDF of the paper titled Misinformation Ex...
