This paper was accepted at the Learning from Evaluating the Evolving LLM Lifecycle workshop at NeurIPS 2025.
Safety evaluation of multimodal foundation models often treats vision and language inputs separately, missing risks from joint interpretation where benign content becomes harmful in combination. Existing approaches also fail to distinguish clearly unsafe content from borderline cases, leading to problematic over-blocking or under-refusal of genuinely harmful content. We present Vision Language Safety Understanding (VLSU), a comprehensive framework to systematically evaluate multimodal… Read More
VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety
This paper was accepted at the Learning from Evaluating the Evolving LLM Lifecycle workshop at NeurIPS 2025.
Safety evaluation of multimodal foundation models often treats vision and language inputs separately, missing risks from joint interpretation where benign content becomes harmful in combination. Existing approaches also fail to distinguish clearly unsafe content from borderline cases, leading to problematic over-blocking or under-refusal of genuinely harmful content. We present Vision Language Safety Understanding (VLSU), a comprehensive framework to systematically evaluate multimodal…
Related Posts
AI Mode is now available in more languages and locations around the world.
Starting today, we’re bringing AI Mode in Google Search to more people around the world,…
The End of the Sync Script: Infrastructure as Intent
There’s an open secret in the world of DevOps: Nobody trusts the CMDB. The Configuration…
7 NumPy Tricks to Vectorize Your Code
You've written Python that processes data in a loop.