A Theory of Post-hoc Debate Judgement

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Xiang Yin, Adam Dejl, Antonio Rago, Lihu Chen, Francesca Toni
• Venue: arXiv: cs.AI
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.19002v1
• PDF: https://arxiv.org/pdf/2608.19002v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

In this research paper published in 2026, Xiang Yin et al. propose a theory of post-hoc debate judgement for agentic AI. The authors discuss the use of debates to improve AI performance and user engagement, and how the outcomes are often determined by external judges like LLMs. They identify properties that debate judgement should satisfy, such as reproducibility, robustness, groundedness, and explainability. The study investigates these properties for claim verification settings and two specific methods of debate judgement: LLM-based judge ideas and formal semantics from computational argumentation. The results suggest similar accuracy performances between the two methods but the former may lack formal guarantees that the latter provides.

What the Document Says

• The paper discusses debates as a methodology for AI improvement, explainability, and user engagement.
• Debate outcomes are often determined post-hoc by external judges, typically LLMs.
• The authors develop a theory of debate judgement applicable to all settings where agents engage in debates.
• They identify formal properties that debate judgement should satisfy: reproducibility, robustness, groundedness, and explainability.
• The study explores these properties for claim verification settings and two specific methods: LLM-based judge ideas and formal semantics from computational argumentation.

Key Concepts Mentioned

• Debates
• AI
• LLMs
• Judgement
• Formal Properties
• Claim Verification

Claims Made by the Document

• The two methods give similar accuracy performances.
• The LLM-based method may lack formal guarantees compared to the formal semantics method.

Evidence Used

• The results of the study suggest similar accuracy performances between the two methods.
• The text mentions that the former method brings formal guarantees that the latter provides.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Debates have recently emerged as a useful methodology for agentic AI to improve performance as well as to aid explainability and user engagement. For example, LLM-empowered agents may debate internally (with themselves) and/or externally (with other agents). In many settings where debates are used, debates' outcomes and resulting outputs are determined post-hoc by external judges, often LLMs. In this paper we develop and test a novel theory of debate judgement applicable to all settings where agents engage in debates by providing pros and cons for their opinions therein. Specifically, we identify a number of formal properties that debate judgement may be required to satisfy in general, as concerns reproducibility, robustness, groundedness and explainability. Then, we explore their satisfaction formally and/or experimentally, for claim verification settings, for two specific alternative debate judgement methods: variants of the LLMs as a judge idea and formal semantics drawn from computational argumentation. We show that the two methods give similar accuracy performances but the former may lack formal guarantees that the latter brings. Overall, our study indicates argumentation semantics as an ideal candidate for principled judges in debate-driven AI.