From Load Tests to Live Streams: Graph Embedding-Based Anomaly Detection in Microservice Architectures

Amazon's Prime Video engineering team has developed a novel graph-based anomaly detection system to identify critical but under-monitored microservices during major live-streaming and video-on-demand events, as detailed in a research paper published on the arXiv preprint server. The system was created to address a significant gap where traditional load tests, while validating overall system capacity, often fail to capture the unique and unpredictable behavior of individual services under real-world traffic conditions, such as during the stream of Thursday Night Football or the premiere of series like The Rings of Power. This advancement aims to proactively ensure platform reliability for millions of concurrent viewers. The core innovation lies in applying unsupervised graph embedding techniques to the complex, interconnected web of microservices that power the streaming platform. The system models the entire architecture as a dynamic graph, where nodes represent individual services and edges represent the communication and dependencies between them. By analyzing the structural embeddings of these nodes during simulated load tests and comparing them to patterns observed during actual live events, the algorithm can pinpoint services whose behavior is "under-represented" in testing scenarios. These are services that may not fail under synthetic stress but could become unexpected bottlenecks or failure points when real user traffic introduces unique interaction patterns and data flows. This approach represents a shift from purely volume-based capacity planning to a more intelligent, topology-aware monitoring strategy. For a platform of Prime Video's scale, where a single malfunctioning microservice can cascade and degrade the viewer experience, such preemptive identification is crucial. The research indicates that this method allows engineers to allocate monitoring resources and robustness improvements more effectively, targeting the specific services most likely to exhibit anomalous behavior during peak events. Ultimately, the goal is to move beyond ensuring the system can handle the load, to guaranteeing it can handle the load in the specific, complex ways real users generate it, thereby minimizing buffering, errors, and downtime during the world's most-watched streaming events.