A Theory of Post-hoc Debate Judgement

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Xiang Yin, Adam Dejl, Antonio Rago, Lihu Chen, Francesca Toni
• Venue: arXiv: cs.AI
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.19002v1
• PDF: https://arxiv.org/pdf/2608.19002v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

In this research, Xiang Yin and colleagues propose a theory of post-hoc judgement for debates used by agentic AI. The goal is to improve the performance, explainability, and user engagement of AI agents through debate. The paper identifies key properties that debate judgement should satisfy, including reproducibility, robustness, groundedness, and explainability. It also compares two methods for making these judgements - LLMs as a judge idea and formal semantics from computational argumentation. The study suggests that while both methods perform similarly, the formal semantics method provides more guarantees.

What the Document Says

• Debates are used by agentic AI for improvement and explainability.
• Debate outcomes are often determined post-hoc by external judges (LLMs).
• The paper proposes a theory of debate judgement applicable to settings where agents provide pros and cons in debates.
• The theory identifies formal properties required for reproducibility, robustness, groundedness, and explainability in debate judgement.
• The study explores these properties for claim verification settings and two specific debate judgement methods: LLMs as a judge idea and formal semantics from computational argumentation.
• Both methods are found to give similar accuracy performances, but the formal semantics method provides more formal guarantees.

Key Concepts Mentioned

• debates
• agentic AI
• LLM
• post-hoc judgement
• reproducibility
• robustness
• groundedness
• explainability
• computational argumentation

Claims Made by the Document

• Debates are useful for agentic AI.
• The proposed theory of debate judgement can be applied to various settings.
• Two methods for debate judgement have been explored and compared.

Evidence Used

• In many settings where debates are used, debates' outcomes and resulting outputs are determined post-hoc by external judges (para 1)

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Debates have recently emerged as a useful methodology for agentic AI to improve performance as well as to aid explainability and user engagement. For example, LLM-empowered agents may debate internally (with themselves) and/or externally (with other agents). In many settings where debates are used, debates' outcomes and resulting outputs are determined post-hoc by external judges, often LLMs. In this paper we develop and test a novel theory of debate judgement applicable to all settings where agents engage in debates by providing pros and cons for their opinions therein. Specifically, we identify a number of formal properties that debate judgement may be required to satisfy in general, as concerns reproducibility, robustness, groundedness and explainability. Then, we explore their satisfaction formally and/or experimentally, for claim verification settings, for two specific alternative debate judgement methods: variants of the LLMs as a judge idea and formal semantics drawn from computational argumentation. We show that the two methods give similar accuracy performances but the former may lack formal guarantees that the latter brings. Overall, our study indicates argumentation semantics as an ideal candidate for principled judges in debate-driven AI.