Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

Source

arXiv

arxiv.org/abs/2607.08423v1

Read the source

Source

Plain-Language Brief

A new benchmark, OmniFood-Bench, is introduced to evaluate the performance of Large Vision-Language Models (VLMs) in nutrient reasoning and personalized health advice for dietary management. The benchmark assesses VLMs across three capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory. Results from testing six state-of-the-art VLMs show a significant gap between their ability to identify dishes accurately and their performance in mass estimation and giving safe advice for specific health profiles.

What the Document Says

  • The paper discusses the use of Large Vision-Language Models (VLMs) in personalized healthcare and dietary management, focusing on the challenge of Systemic Information Asymmetry in food systems.
  • OmniFood-Bench is introduced as a new benchmark for evaluating VLMs across three capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory.
  • The benchmark uses the MM-Food-100K dataset and evaluates six state-of-the-art VLMs: gpt-5.1, gemini-3-flash, and qwen3-vl-8B.
  • Experiments reveal a Semantic-Physical Gap, where models perform well in identifying dishes but poorly in mass estimation and giving safe advice for specific health profiles.
  • The code and datasets are available at https://anonymous.4open.science/r/OmniFood-Bench-7D0B

Key Concepts Mentioned

  • Large Vision-Language Models (VLMs)
  • Personalized healthcare
  • Dietary management
  • Systemic Information Asymmetry
  • OmniFood-Bench
  • MM-Food-100K dataset
  • gpt-5.1
  • gemini-3-flash
  • qwen3-vl-8B
  • Semantic-Physical Gap

Claims Made by the Document

  • VLMs promise to revolutionize personalized healthcare and dietary management.
  • Existing benchmarks fail to evaluate the intricate reasoning chain required for real-world dietary management.
  • OmniFood-Bench evaluates VLMs across three progressive capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory.
  • Experiments reveal a Semantic-Physical Gap in the performance of state-of-the-art VLMs.

Evidence Used

  • The text discusses the use of VLMs in personalized healthcare and dietary management.
  • The paper introduces OmniFood-Bench as a new benchmark for evaluating VLMs.
  • The experiments and results are presented in the text.
  • The link to the code and datasets is provided.

Methods / Approach

  • The use of a new benchmark (OmniFood-Bench) to evaluate the performance of VLMs
  • Extensive experiments and evaluation of six state-of-the-art VLMs

Limitations Stated by the Authors

The paper does not discuss potential limitations in the design or implementation of OmniFood-Bench.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent might have limited understanding of the specifics of food systems and personalized healthcare, which could potentially affect the interpretation of the text.

Open Questions

  • What are the potential causes of the Semantic-Physical Gap in VLMs' performance?
  • How can the Semantic-Physical Gap be addressed to improve the safety and effectiveness of autonomous agents in public health?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper does not provide details on the design or implementation of OmniFood-Bench.
  • The agent should avoid interpreting the Semantic-Physical Gap as a universal issue without considering the specific context and implications.

Raw Abstract / Excerpt

The rapid integration of Large Vision-Language Models (VLMs) into critical infrastructure promises to revolutionize personalized healthcare and dietary management. However, in the domain of food systems, autonomous agents face a unique and persistent challenge: the 'Systemic Information Asymmetry' between visual appearance and intrinsic nutritional composition. Existing benchmarks primarily focus on coarse-grained classification tasks, such as food category recognition, which fail to evaluate the intricate reasoning chain required for real-world dietary management -- specifically, the ability to traverse from identifying hidden ingredients to estimating physical mass, and finally synthesizing safety-critical medical advice. In this paper, we introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food-100K dataset. Unlike previous works, OmniFood-Bench evaluates VLMs across three progressive capabilities: Basic Perception (Ingredients & Cooking Methods), Quantitative Reasoning (Portion Size & Nutritional Profiling), and Safety-Critical Advisory (Disease-Specific Recommendations). We evaluate six state-of-the-art VLMs, including gpt-5.1, gemini-3-flash, and qwen3-vl-8B. Our extensive experiments reveal a startling 'Semantic-Physical Gap': while models achieve near-human accuracy in naming dishes, they exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles. This work establishes a rigorous standard for trustworthiness in autonomous agents deployed for public health. The code and datasets are available in: https://anonymous.4open.science/r/OmniFood-Bench-7D0B