Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

Source

arXiv

arxiv.org/abs/2607.24601v1

Read the source

Source

Plain-Language Brief

A study investigates how the level of explanation in AI-assisted code reviews affects developer trust and agreement with AI recommendations. The results show that full explanations lead to higher perceived trust but not the highest agreement, moderate explanations achieve the highest agreement, and no explanations result in the lowest trust and agreement.

What the Document Says

  • The study investigates the influence of Explainable AI (XAI) on developer trust in AI-assisted code reviews.
  • A user study was conducted with 34 participants comparing three LLM-based code review systems with varying levels of XAI support: detailed explanations and review feedback (Condition A), review feedback only (Condition B), and no explanations (Condition C).
  • Participants reviewed real-world code change requests alongside the AI-generated reviews.
  • Trust perceptions, agreement with the AI recommendation, the reasoning given for each decision, and the time taken were measured.
  • The level of explanation significantly influences both trust and agreement with AI recommendations, but in different ways. Full explanations yield the highest perceived trust but not the highest agreement, moderate explanations achieve the highest agreement, and no explanations result in the lowest trust and agreement.
  • Explanation level did not significantly affect review time.
  • The most commonly cited reasons for decisions were code readability and correctness.

Key Concepts Mentioned

  • Large language models (LLMs)
  • code review
  • Explainable AI (XAI)
  • developer trust
  • AI-assisted software development

Claims Made by the Document

  • The level of explanation influences both trust and agreement with AI recommendations.
  • Full explanations yield the highest perceived trust but not the highest agreement.
  • Moderate explanations achieve the highest agreement.
  • No explanations result in the lowest trust and agreement.

Evidence Used

  • Data from a user study comparing three LLM-based code review systems with varying levels of XAI support
  • Measurements of trust perceptions, agreement with AI recommendation, reasoning given for decisions, and time taken

Methods / Approach

  • Within-subjects user study
  • Comparison of three LLM-based code review systems

Limitations Stated by the Authors

  • The study only investigated the impact of XAI on a specific number (34) of participants.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The findings may not be generalizable to other populations or scenarios outside of the specific conditions tested.
  • Additional studies with larger sample sizes and different LLM-based code review systems could provide more robust results.

Open Questions

  • What is the long-term impact of XAI on developer trust in AI-assisted code reviews?
  • How can XAI be further optimized to improve both trust and agreement with AI recommendations?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The study did not mention any specific large language models used in the experiment.

Raw Abstract / Excerpt

Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to understand. Developers struggle to assess the validity of LLM-generated reviews, making it difficult to gauge how much trust to place in them. The role of Explainable AI (XAI) in code review and its impact on trust remain underexplored. Objective: We study the influence of XAI on developer trust in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants, comparing three LLM-based code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants reviewed real-world code change requests alongside the AI-generated reviews. We measured trust perceptions, agreement with the AI recommendation, the reasoning given for each decision, and the time taken. Results: The level of explanation significantly influences both trust and agreement with AI recommendations, but in different ways. Full explanations (A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement, whereas moderate explanations (B) achieve the highest agreement (89.22%). This could suggest that more explanation prompts developers to question AI recommendations more frequently. No explanations (C) results in the lowest trust and agreement. Explanation level did not significantly affect review time. The most commonly cited reasons for decisions were code readability and correctness. Conclusion: Incorporating XAI into code review significantly changes trust perceptions and agreement with AI recommendations. These results inform the design and evaluation of trustworthy AI-based code review systems, as well as studies on the human factors of AI-assisted software development.