Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

Source

arXiv

arxiv.org/abs/2607.24601v1

Read the source

Source

Plain-Language Brief

A study investigates the impact of Explainable AI (XAI) on developers' trust in AI-assisted code reviews. The research compared three LLM-based code review systems with varying levels of XAI support. Results show that the level of explanation significantly influences both trust and agreement with AI recommendations, but not review time. The most commonly cited reasons for decisions were code readability and correctness.

What the Document Says

  • The study focuses on the influence of Explainable AI (XAI) on developers' trust in AI-assisted code reviews.
  • Three LLM-based code review systems with varying levels of XAI support were compared: detailed explanation and review feedback (Condition A), review feedback only (Condition B), and no explanations (Condition C).
  • Participants reviewed real-world code change requests alongside the AI-generated reviews.
  • Trust perceptions, agreement with the AI recommendation, reasoning given for each decision, and time taken were measured.
  • The level of explanation significantly influences both trust and agreement with AI recommendations, but not review time.
  • Full explanations (A) yielded the highest perceived trust but not the highest agreement. Moderate explanations (B) achieved the highest agreement.
  • No explanations (C) resulted in the lowest trust and agreement.
  • The most commonly cited reasons for decisions were code readability and correctness.
  • The study concludes that incorporating XAI into code review significantly changes trust perceptions and agreement with AI recommendations.

Key Concepts Mentioned

  • Explainable AI (XAI)
  • Large language models (LLMs)
  • code review
  • developer trust
  • within-subjects user study

Claims Made by the Document

  • The level of explanation significantly influences both trust and agreement with AI recommendations.
  • Full explanations yield the highest perceived trust but not the highest agreement.
  • Moderate explanations achieve the highest agreement.

Evidence Used

  • Results section

Methods / Approach

  • Conducted a within-subjects user study comparing three LLM-based code review systems with varying levels of XAI support.

Limitations Stated by the Authors

  • The study only involved 34 participants.
  • The research was based on one specific type of AI-assisted code review system (LLMs)

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The findings may not be generalizable to other types of AI or contexts.
  • A larger sample size might provide more reliable results.

Open Questions

  • How does XAI impact the long-term trust in AI-assisted code reviews?
  • Can this study's findings be replicated with other types of AI or contexts?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The study measures trust perceptions and agreement, but it does not delve into the nuances of these concepts.
  • It is unclear whether the participants were familiar with XAI or its impact on code review before the study.

Raw Abstract / Excerpt

Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to understand. Developers struggle to assess the validity of LLM-generated reviews, making it difficult to gauge how much trust to place in them. The role of Explainable AI (XAI) in code review and its impact on trust remain underexplored. Objective: We study the influence of XAI on developer trust in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants, comparing three LLM-based code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants reviewed real-world code change requests alongside the AI-generated reviews. We measured trust perceptions, agreement with the AI recommendation, the reasoning given for each decision, and the time taken. Results: The level of explanation significantly influences both trust and agreement with AI recommendations, but in different ways. Full explanations (A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement, whereas moderate explanations (B) achieve the highest agreement (89.22%). This could suggest that more explanation prompts developers to question AI recommendations more frequently. No explanations (C) results in the lowest trust and agreement. Explanation level did not significantly affect review time. The most commonly cited reasons for decisions were code readability and correctness. Conclusion: Incorporating XAI into code review significantly changes trust perceptions and agreement with AI recommendations. These results inform the design and evaluation of trustworthy AI-based code review systems, as well as studies on the human factors of AI-assisted software development.