VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation

Researchers Seongheon Park, Changdae Oh, Hyeong Kyu Choi, Xuefeng Du, and Sharon Li introduced VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation, in a paper submitted to arXiv on February 24, 2026, addressing the critical issue of hallucination in Large Vision-Language Models that limits their safe deployment in real-world applications. The research team developed this innovative approach to overcome the limitations of existing self-evaluation methods that rely heavily on language priors, making them unsuitable for evaluating vision-conditioned predictions. VAUQ represents a significant advancement in how AI systems can assess their own reliability when processing visual information. The VAUQ framework introduces two key innovations: the Image-Information Score, which captures the reduction in predictive uncertainty attributable to visual input, and an unsupervised core-region masking strategy that amplifies the influence of salient regions in images. By combining predictive entropy with this core-masked Image-Information Score, the researchers created a training-free scoring function that reliably reflects answer correctness without requiring additional training data or computational resources. This approach allows the model to determine how much its predictions depend on actual visual evidence versus internal assumptions or biases. Comprehensive experiments conducted by the research team demonstrate that VAUQ consistently outperforms existing self-evaluation methods across multiple datasets, establishing a new standard for assessing the reliability of vision-language models. The framework addresses a critical challenge in AI safety and reliability, particularly as these models become increasingly integrated into high-stakes applications like medical diagnosis, autonomous systems, and content moderation. By enabling more accurate self-assessment, VAUQ contributes to the responsible deployment of advanced AI systems that must process both visual and linguistic information.