SignVLA: A Gloss-Free Vision-Language-Action Framework for Real-Time Sign Language-Guided Robotic Manipulation

A team of researchers led by Xinyu Tan and Ningwei Bai introduced SignVLA, a groundbreaking gloss-free Vision-Language-Action framework for real-time sign language-guided robotic manipulation in a paper submitted to arXiv on February 26, 2026. Unlike conventional approaches that rely on gloss annotations as intermediate supervision, the proposed system directly maps visual sign gestures to semantic instructions, reducing annotation costs and avoiding information loss. The framework focuses on alphabet-level finger-spelling interfaces that provide robust, low-latency communication channels for robotic control in safety-critical environments. The SignVLA framework represents a significant advancement in human-robot interaction by eliminating the need for gloss annotations, which have traditionally been used as intermediate supervision in sign language recognition systems. By directly mapping visual sign gestures to semantic instructions, the researchers have created a more natural and scalable multimodal interaction method. The pipeline transforms continuous gesture streams into coherent language commands through geometric normalization, temporal smoothing, and lexical refinement, ensuring stable and consistent interaction across diverse scenarios. The researchers specifically focused on alphabet-level finger-spelling interfaces because they offer improved reliability, interpretability, and deployment feasibility compared to large-scale continuous sign language recognition, particularly in safety-critical embodied environments. This approach provides a robust and low-latency communication channel for robotic control. Furthermore, the framework is designed with future expansion in mind, supporting the integration of transformer-based gloss-free sign language models that could enable scalable word-level and sentence-level semantic understanding. Experimental results demonstrate the system's effectiveness in grounding sign-derived instructions into precise robotic actions, highlighting its potential to advance accessible, scalable, and multimodal embodied intelligence.