From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
#LLM #multi‑agent systems #failure attribution #causal graph #hierarchical model #oracle‑guided backtracking #counterfactual attribution #CHIEF #Who&When benchmark #cs.AI
📌 Key Takeaways
- The paper starts by critiquing flat‑sequence failure attribution methods in LLM‑based multi‑agent systems for obscuring causal links and responsibility.
- It introduces CHIEF, a framework that converts chaotic MAS execution traces into a structured hierarchical causal graph.
- CHIEF incorporates hierarchical oracle‑guided backtracking to prune the causal search space, and a counterfactual attribution strategy to filter genuine root causes from downstream symptoms.
- Experiments on the Who&When benchmark demonstrate that CHIEF surpasses eight state‑of‑the‑art baselines on both agent‑level and step‑level accuracy.
- Ablation studies confirm that each component—hierarchical graph construction, oracle backtracking, and counterfactual screening—contributes significantly to overall performance.
📖 Full Retelling
Yawen Wang, Wenjie Wu, Junjie Wang, and Qing Wang, researchers in the field of Artificial Intelligence, have proposed CHIEF (a novel hierarchical failure attribution framework) to address the fragility and opaque failure mechanisms of LLM‑powered Multi‑Agent Systems (MAS). Published on the arXiv repository on 27 February 2026, the work targets the limitations of existing flat‑sequence log‑based attribution methods, which obscure causal links and responsibility boundaries within MAS. By transforming chaotic execution trajectories into structured hierarchical causal graphs and employing oracle‑guided backtracking alongside counterfactual attribution, CHIEF seeks to deliver clearer root‑cause identification and improved observability for MAS failures.
🏷️ Themes
Failure Attribution, Artificial Intelligence, Explainability, Causal Modeling, Large Language Models, Multi‑Agent Systems, Software Engineering
Entity Intersection Graph
No entity connections available yet for this article.
Original Source
--> Computer Science > Artificial Intelligence arXiv:2602.23701 [Submitted on 27 Feb 2026] Title: From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems Authors: Yawen Wang , Wenjie Wu , Junjie Wang , Qing Wang View a PDF of the paper titled From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems, by Yawen Wang and 3 other authors View PDF HTML Abstract: LLM-powered Multi-Agent Systems have demonstrated remarkable capabilities in complex domains but suffer from inherent fragility and opaque failure mechanisms. Existing failure attribution methods, whether relying on direct prompting, costly replays, or supervised fine-tuning, typically treat execution logs as flat sequences. This linear perspective fails to disentangle the intricate causal links inherent to MAS, leading to weak observability and ambiguous responsibility boundaries. To address these challenges, we propose CHIEF, a novel framework that transforms chaotic trajectories into a structured hierarchical causal graph. It then employs hierarchical oracle-guided backtracking to efficiently prune the search space via sybthesized virtual oracles. Finally, it implements counterfactual attribution via a progressive causal screening strategy to rigorously distinguish true root causes from propagated symptoms. Experiments on Who&When benchmark show that CHIEF outperforms eight strong and state-of-the-art baselines on both agent- and step-level accuracy. Ablation studies further confirm the critical role of each proposed module. Subjects: Artificial Intelligence (cs.AI) ; Software Engineering (cs.SE) Cite as: arXiv:2602.23701 [cs.AI] (or arXiv:2602.23701v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2602.23701 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Wenjie Wu [ view email ] [v1] Fri, 27 Feb 2026 06:08:42 UTC (1,505 KB) Full-text links: Access Paper: View a PDF...
Read full article at source