Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

Source

arXiv

arxiv.org/abs/2607.08423v1

Read the source

Source

Plain-Language Brief

The paper introduces OmniFood-Bench, a comprehensive benchmark for evaluating the performance of Vision-Language Models (VLMs) in personalized healthcare and dietary management. The benchmark focuses on three capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory. The authors tested six state-of-the-art VLMs and found a significant gap between the models' ability to identify dishes and their accuracy in mass estimation and providing safe advice for high-risk profiles.

What the Document Says

  • The paper discusses the integration of Large Vision-Language Models (VLMs) into healthcare and dietary management.
  • A challenge faced by autonomous agents in food systems is the 'Systemic Information Asymmetry' between visual appearance and intrinsic nutritional composition.
  • Existing benchmarks focus on coarse-grained classification tasks, which do not evaluate the reasoning chain required for real-world dietary management.
  • The paper introduces OmniFood-Bench, a benchmark constructed from MM-Food-100K dataset that evaluates VLMs across three progressive capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory.
  • Six state-of-the-art VLMs were tested: gpt-5.1, gemini-3-flash, and qwen3-vl-8B.
  • The experiments revealed a 'Semantic-Physical Gap': models achieved near-human accuracy in naming dishes but had catastrophic failures in mass estimation and often provided incorrect advice for high-risk diabetic profiles.
  • The work sets a standard for trustworthiness in autonomous agents deployed for public health.

Key Concepts Mentioned

  • Vision-Language Models (VLMs)
  • Systemic Information Asymmetry
  • OmniFood-Bench
  • MM-Food-100K dataset
  • Personalized healthcare
  • Dietary management
  • Semantic-Physical Gap

Claims Made by the Document

  • Existing benchmarks do not evaluate the reasoning chain required for real-world dietary management.
  • The 'Semantic-Physical Gap' exists in VLMs.

Evidence Used

  • The text mentions that existing benchmarks focus on coarse-grained classification tasks, which do not evaluate the intricate reasoning chain required for real-world dietary management.
  • The text states that the experiments revealed a 'Semantic-Physical Gap' in VLMs.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

The rapid integration of Large Vision-Language Models (VLMs) into critical infrastructure promises to revolutionize personalized healthcare and dietary management. However, in the domain of food systems, autonomous agents face a unique and persistent challenge: the 'Systemic Information Asymmetry' between visual appearance and intrinsic nutritional composition. Existing benchmarks primarily focus on coarse-grained classification tasks, such as food category recognition, which fail to evaluate the intricate reasoning chain required for real-world dietary management -- specifically, the ability to traverse from identifying hidden ingredients to estimating physical mass, and finally synthesizing safety-critical medical advice. In this paper, we introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food-100K dataset. Unlike previous works, OmniFood-Bench evaluates VLMs across three progressive capabilities: Basic Perception (Ingredients & Cooking Methods), Quantitative Reasoning (Portion Size & Nutritional Profiling), and Safety-Critical Advisory (Disease-Specific Recommendations). We evaluate six state-of-the-art VLMs, including gpt-5.1, gemini-3-flash, and qwen3-vl-8B. Our extensive experiments reveal a startling 'Semantic-Physical Gap': while models achieve near-human accuracy in naming dishes, they exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles. This work establishes a rigorous standard for trustworthiness in autonomous agents deployed for public health. The code and datasets are available in: https://anonymous.4open.science/r/OmniFood-Bench-7D0B