AIR: Adaptive Interleaved Reasoning with Code in MLLMs

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Cong Han, Xiaohan Lan, Haibo Qiu, Yujie Zhong
• Venue: arXiv: cs.CV
• Published: 2026-06-22
• Collected: 2026-06-23
• URL: http://arxiv.org/abs/2606.23678v1
• PDF: https://arxiv.org/pdf/2606.23678v1
• DOI: UNKNOWN

Plain-Language Brief

The paper presents a new approach to enhance multimodal large language models (MLLMs) by giving them the ability to reason adaptively with code, particularly in numerical computation tasks. This is achieved through extended reinforcement learning training on complex numerical tasks. The proposed solution includes a two-stage data construction pipeline, data filtering strategies for RL dataset curation, and an adaptive tool-invocation strategy using a group-constrained reward function.

What the Document Says

• The paper focuses on interleaved reasoning with code to improve MLLMs following the OpenAI o3 paradigm shift.
• Existing approaches rely on predefined heuristics for visual manipulation and cannot address numerical computation problems due to their focus on visual operations.
• The new approach empowers MLLMs with adaptive interleaved reasoning capabilities through extended reinforcement learning training on code-augmented complex numerical tasks.
• The solution consists of a two-stage cold-start data construction pipeline, data filtering strategies for RL dataset curation, and an adaptive tool-invocation strategy using a group-constrained reward function.

Key Concepts Mentioned

• MLLMs
• interleaved reasoning with code
• reinforcement learning
• numerical computation tasks

Claims Made by the Document

• The new approach improves performance on evaluation benchmarks by an average of 6.1 percentage points.
• The accuracy for interleaved reasoning samples increases by 9.9 percentage points.
• The overall success rate of tool-use exceeds 95%.

Evidence Used

• Improvement in performance is demonstrated through extensive experiments.

Methods / Approach

• extended reinforcement learning training
• two-stage data construction pipeline
• data filtering strategies
• adaptive tool-invocation strategy

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Following the paradigm shift initiated by OpenAI o3, interleaved reasoning with code to enhance multimodal large language models (MLLMs) has become a pivotal research frontier. The existing literature focuses primarily on tool-use within vision-perception tasks. However, such approaches typically rely on predefined heuristics for visual manipulation and are inherently incapable of addressing numerical computation problems due to their exclusive focus on visual operations. This paper empowers MLLMs with adaptive interleaved reasoning capabilities through extended reinforcement learning training on code-augmented complex numerical computation tasks. To this end, we propose a comprehensive three-component solution consisting of: a two-stage cold-start data construction pipeline, data filtering strategies for RL dataset curation, and an adaptive tool-invocation strategy leveraging a group-constrained reward function for interleaved reasoning trajectories. Extensive experiments demonstrate that after Reinforcement Learning training with the group-constrained reward function, performance improves by an average of 6.1 percentage points (pp) on evaluation benchmarks. Specifically, the accuracy for interleaved reasoning samples increases by 9.9 pp, and the overall success rate of tool-use exceeds 95%. Our data and code are available at: https://github.com/CongHan0808/AIR.git.