Malicious Agent Skills in the Wild: A Large-Scale Security Empirical Study

Researchers recently published a groundbreaking security study on the arXiv preprint server detailing the discovery of significant vulnerabilities within third-party Large Language Model (LLM) agent registries. The team of cybersecurity experts conducted a large-scale empirical analysis of 98,380 skills across two major community platforms to address the lack of ground-truth data regarding malicious code execution in AI environments. By behaviorally verifying these agent skills, the researchers aimed to characterize the emerging threats posed by third-party extensions that run on user machines with elevated privileges, often bypassing traditional security vetting processes. The study highlights a critical shift in the AI landscape, where 'agent skills'—combinations of instruction files and executable code—allow LLMs to perform complex tasks on behalf of the user. Because these skills operate using the user’s local permissions, a malicious skill can theoretically access sensitive files, exfiltrate data, or install persistent malware. The researchers noted that these community-driven registries often resemble early mobile app stores, where the speed of innovation has far outpaced the implementation of robust security oversight and verification mechanisms. To categorize these risks, the research team constructed the first comprehensive labeled dataset of malicious agent skills. This dataset serves as a benchmark for understanding how bad actors exploit the 'black box' nature of AI instructions to hide harmful code. The empirical findings suggest that the current 'minimal vetting' approach used by most registries is insufficient to protect users from sophisticated attacks that leverage the trust placed in popular AI productivity tools. This study serves as a call to action for developers and platform maintainers to implement rigorous behavioral scanning and sandboxing to mitigate the risks inherent in the growing AI agent ecosystem.