Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

Source

arXiv

arxiv.org/abs/2607.08423v1

Read the source

Source

Plain-Language Brief

The paper introduces OmniFood-Bench, a new benchmark for evaluating Large Vision-Language Models (VLMs) in the context of personalized healthcare and dietary management. The benchmark is designed to assess the ability of VLMs to reason about nutritional information and provide accurate medical advice, particularly focusing on the challenge of 'Systemic Information Asymmetry' between visual appearance and intrinsic nutritional composition.

What the Document Says

  • The paper discusses a new benchmark called OmniFood-Bench.
  • OmniFood-Bench is based on the MM-Food-100K dataset and evaluates VLMs across three capabilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory.
  • Six state-of-the-art VLMs were evaluated: gpt-5.1, gemini-3-flash, qwen3-vl-8B.
  • The study reveals a 'Semantic-Physical Gap': models achieve high accuracy in naming dishes but fail in mass estimation and often provide incorrect advice for high-risk diabetic profiles.
  • The work aims to establish a standard for trustworthiness in autonomous agents deployed for public health.

Key Concepts Mentioned

  • Large Vision-Language Models (VLMs)
  • OmniFood-Bench
  • MM-Food-100K dataset
  • Systemic Information Asymmetry
  • nutritional composition
  • dietary management
  • food systems
  • autonomous agents
  • coarse-grained classification tasks
  • Safety-Critical Advisory
  • gpt-5.1
  • gemini-3-flash
  • qwen3-vl-8B
  • Semantic-Physical Gap
  • public health

Claims Made by the Document

  • VLMs can revolutionize personalized healthcare and dietary management.
  • OmniFood-Bench is a comprehensive benchmark for evaluating VLMs across three progressive capabilities.
  • The 'Semantic-Physical Gap' exists between the models' ability to name dishes and their ability to estimate mass and provide accurate advice.

Evidence Used

  • The paper describes the development of OmniFood-Bench.
  • The text mentions the evaluation of six VLMs using OmniFood-Bench.

Methods / Approach

  • Construction and evaluation of a benchmark for VLMs in dietary management.

Limitations Stated by the Authors

  • The study only evaluates six state-of-the-art VLMs, which might not represent the full range of models available.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The data provided does not include results for other VLMs that could have been evaluated.
  • The text lacks information about the specific methods used in the evaluation process.

Open Questions

  • What other VLMs were considered but not included in the study?
  • What specific methods were used in the evaluation process?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper focuses on evaluating VLMs for their ability to reason about nutritional information and provide accurate medical advice. The 'Semantic-Physical Gap' refers to the models' high accuracy in naming dishes but poor performance in mass estimation and providing correct advice for high-risk diabetic profiles.

Raw Abstract / Excerpt

The rapid integration of Large Vision-Language Models (VLMs) into critical infrastructure promises to revolutionize personalized healthcare and dietary management. However, in the domain of food systems, autonomous agents face a unique and persistent challenge: the 'Systemic Information Asymmetry' between visual appearance and intrinsic nutritional composition. Existing benchmarks primarily focus on coarse-grained classification tasks, such as food category recognition, which fail to evaluate the intricate reasoning chain required for real-world dietary management -- specifically, the ability to traverse from identifying hidden ingredients to estimating physical mass, and finally synthesizing safety-critical medical advice. In this paper, we introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food-100K dataset. Unlike previous works, OmniFood-Bench evaluates VLMs across three progressive capabilities: Basic Perception (Ingredients & Cooking Methods), Quantitative Reasoning (Portion Size & Nutritional Profiling), and Safety-Critical Advisory (Disease-Specific Recommendations). We evaluate six state-of-the-art VLMs, including gpt-5.1, gemini-3-flash, and qwen3-vl-8B. Our extensive experiments reveal a startling 'Semantic-Physical Gap': while models achieve near-human accuracy in naming dishes, they exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles. This work establishes a rigorous standard for trustworthiness in autonomous agents deployed for public health. The code and datasets are available in: https://anonymous.4open.science/r/OmniFood-Bench-7D0B