Halluverse-M^3: A multitask multilingual benchmark for hallucination in LLMs

A team of researchers introduced a groundbreaking multitask multilingual benchmark named Halluverse-M^3 on the arXiv preprint server in February 2025 to address the persistent challenge of hallucinations in Large Language Models (LLMs). This new dataset was developed to provide a systematic framework for evaluating factual consistency and generative errors across a diverse range of languages and tasks, moving beyond the traditional English-centric evaluation methods that currently dominate the industry. The initiative stems from an urgent need to understand how AI models behave when operating outside of their primary training language, where the risk of generating false or misleading information significantly increases. While contemporary LLMs have demonstrated remarkable proficiency in English-based benchmarks, their reliability often falters in multilingual and generative settings. Halluverse-M^3 targets these vulnerabilities by categorizing different types of hallucinations, allowing developers to pinpoint specific weaknesses in model logic or knowledge retrieval. By offering a multi-dimensional perspective, the benchmark helps researchers distinguish between translation errors, factual fabrications, and contextual inconsistencies that occur when a model attempts to synthesize information across different cultural and linguistic frameworks. The release of Halluverse-M^3 signifies a shift toward more inclusive and rigorous AI safety standards. As Large Language Models are integrated into global services—from automated translation to international customer support—the ability to measure and mitigate non-English hallucinations becomes critical for user safety and data integrity. The dataset provides the necessary tools for a more nuanced analysis of how these models process information, ensuring that future iterations of AI are not only more capable but also more factually grounded on a global scale.