Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia

A team of researchers has introduced 'Mini-Mafia,' a specialized testing framework designed to evaluate the social intelligence of Large Language Models (LLMs) through a simplified version of the popular social deduction game. The study, detailed in a paper updated on the arXiv preprint server in late 2024, utilizes a four-player game structure consisting of one mafioso, one detective, and two villagers to measure how effectively AI agents can handle information asymmetry and theory-of-mind reasoning. This experimentation aims to bridge the gap between static AI performance and the complex, deceptive multi-agent environments found in real-world human interactions. The 'Mini-Mafia' variant was specifically constructed to provide a controlled environment for systematic study, moving away from more complex, larger versions of the game that can be computationally expensive or difficult to analyze. By limiting the player count and roles, the researchers can focus on the critical mechanics of deception and detection. In this setup, the 'informed' agent (the mafioso) must deceive others to survive, while the 'uninformed' agents (the townsfolk) must use logic and behavioral analysis to reveal the hidden threat. Central to the research is the concept of 'Theory of Mind,' which refers to the ability to attribute mental states—such as beliefs, intents, and knowledge—to oneself and others. Because Mafia relies heavily on understanding what other players know versus what they are pretending to know, it serves as a rigorous benchmark for LLMs. The study explores whether current models can maintain consistent personas, detect lies in their peers, and strategically disclose or withhold information to achieve a group or individual goal. The findings from this research have significant implications for the development of more sophisticated AI assistants and autonomous agents. As LLMs are increasingly integrated into collaborative workspaces and social platforms, understanding their capacity for strategic reasoning and their vulnerability to or aptitude for deception is vital. 'Mini-Mafia' represents a step toward quantifiable metrics for social artificial intelligence, allowing developers to see where models fail in nuanced, high-stakes communication scenarios.