Evaluating Large Language Models on Solved and Unsolved Problems in Graph Theory: Implications for Computing Education

Researchers recently published a new technical study on the arXiv preprint server on February 10, 2025, evaluating the mathematical reasoning capabilities of Large Language Models (LLMs) within the specialized field of graph theory. The study was initiated to determine how effectively these artificial intelligence tools support rigorous academic learning as students increasingly rely on them for advanced computer science coursework. By testing the models against both established theorems and unsolved conjectures, the authors sought to uncover the specific limitations of LLMs in handling complex, abstract logical problems that lack a simple, pre-existing solution in their training data. The analysis specifically focused on the performance of LLMs regarding the "gracefulness" of line graphs, comparing their ability to solve a known problem against an open, unsolved challenge in graph theory. This distinction is critical for educators, as LLMs frequently excel at reproducing known proofs through pattern matching but often struggle or hallucinate when confronted with novel mathematical logic. The researchers observed that while the models could articulate basic principles of graph theory, their capacity for deep structural reasoning remains inconsistent, raising concerns about their use as primary pedagogical tools in graduate-level computer science programs. Beyond simple performance metrics, the paper discusses the broader implications for computing education and the necessity of academic integrity in the age of AI. As LLMs become integrated into undergraduate curricula, there is a growing risk that students may accept plausible-sounding but incorrect logical steps as fact. The study suggests that for LLMs to become reliable assistants in mathematical research, current architectures must move beyond statistical prediction toward more robust symbolic reasoning systems. This investigation serves as a cautionary guide for curriculum designers who are currently navigating the transition toward AI-saturated learning environments.