The Trojan horse in the courtroom: The impact of flawed AI advice on legal discretionary decision‐making

Abstract

Purpose

AI-assisted adjudication is increasingly embedded in judicial decision-making, yet whether legal professionals and laypersons can independently guard against overreliance remains contested. This study examines how people with varying legal expertise respond to demonstrably flawed AI recommendations and whether reliance differs across legal-error types.

Methods

Legal professionals, law students and laypersons each adjudicated one criminal and one civil case after random assignment to correct or incorrect AI-generated recommendations. Incorrect advice involved two error types: misapplication of statutory provisions and erroneous citation of judicial precedents. Responses were coded using two independent adoption indicators. Explicit adoption captured direct incorporation of the AI-provided legal basis; implicit adoption captured a final decision materially aligned with the AI recommendation without requiring explicit incorporation of that basis.

Results

Explicit adoption showed a floor effect: participants rarely reproduced, cited or endorsed the AI-provided legal basis, regardless of advice accuracy or legal expertise. Implicit adoption was significantly less frequent for erroneous than correct advice. Legal expertise showed neither a significant main effect nor an interaction with advice condition, and no detectable expertise differences emerged in either adoption outcome. Subjective trust similarly declined significantly in response to AI errors, with no detectable main effect of legal expertise or interaction with advice condition.

Conclusions

These findings are consistent with selective adoption under the tested conditions: across expertise levels, participants aligned less often with erroneous than with correct AI recommendations. Nevertheless, further scrutiny is needed of how AI systems influence legal reasoning in practice.