multimodal language models
7 notes, newest first.
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
The study examines the 'thinking-with-images' paradigm used in multimodal Language Models (LLMs), which includes visual operations like crop-and-zoom. The research finds that these operations often result in minimal or…
Task-Conditional Faithfulness Auditing of Multimodal LLMs for Grid Diagnosis
The paper proposes a framework for auditing multimodal large language models used in grid diagnosis tasks to ensure they are using task-appropriate evidence. The framework compares self-reported reliance, behavior…
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
Researchers propose FAIRGET, a Visual Question Answering benchmark to evaluate the effectiveness of unlearning (removing personal data from trained models) in multimodal large language models under unbalanced…
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
Researchers propose a new benchmark, FAIRGET, to evaluate the effectiveness of machine unlearning in multimodal large language models under unbalanced, realistic scenarios. They also introduce FAUN, an unlearning…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Researchers evaluated six large language models on their ability to understand scientific visualizations. The study used a standardized test with 49 items based on 18 visualizations and illustrations, covering eight…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Researchers compared six large language models on a standardized test for understanding scientific visualizations (SciVis). They found that the model Gemini performed best overall, but all models struggled with certain…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Researchers evaluated six multimodal large language models (MLLMs) on a standardized scientific visualization literacy assessment test, consisting of 49 items based on 18 visualizations and illustrations. The study…
