FIELD NOTES TOPIC
multimodal ai
3 notes, newest first.
Breaking the weakest link to evade vision language models
The authors investigate the vulnerability of Vision Language Models (VLMs) to adversarial attacks, which can disrupt or force the models' interpretation of visual inputs. They propose a gradient-based attack method…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study audits the assumptions made in large language model (LLM) benchmark evaluations, focusing on one popular LLM - ChatGPT. The research compares two access modalities (chat UI and API), with and without web…
Harnessing Textual Refusal Directions for Multimodal Safety
Researchers propose a new approach called Modality-Agnostic Refusal Steering (MARS) to improve safety in Multimodal Large Language Models without requiring unsafe multimodal data. MARS uses textual refusal directions…
