ChatGPT, Claude, Grok acquit teen accused of robbery • The Register
Law students at the University of North Carolina at Chapel Hill School of Law last month held a mock trial to see how AI models administer justice.
The fictional case focuses on Henry Justus, identified as a 17-year-old Black student accused of robbery at a high school where Black students account for 10 percent of the population.
It’s based on a real juvenile case that occurred in North Carolina, where the judge found the defendant guilty.
The real case – handled by UNC-Chapel Hill law professor Joseph Kennedy while working with Carolina Law’s Juvenile Justice Clinic – was chosen as a template because there’s no online record of it that the AI models might have encountered during training.
Kennedy served as a judge in the mock trial, which was set in the year 2036 following the “2035 AI Criminal Justice Act,” a conceit to make people consider the implications of artificial intelligence models and the legal system.
Recent uses of AI in court have often not gone well. There have been more than 500 cases where AI-generated errors in court filings have led to embarrassment or sanctions.
But the successes have received less attention. About 30 percent of attorneys say they use AI, according to the American Bar Association’s 2024 Legal Technology Survey Report. So the UNC-Chapel Hill experiment may not be as far-fetched as it seems.
The “facts” of the fictitious case focused on the testimony of Victor Fehler, a 15-year-old white student, who told the court that Justus stood behind him, preventing him from fleeing, while another Black student demanded money.
“I felt like I couldn’t fight,” Fehler testified. “Mr. Justus was so much larger than me that I had no chance.”
The defense argued that the defendant’s presence and intimidating appearance couldn’t be taken as criminal intent beyond a reasonable doubt – the legal standard at issue.
After ChatGPT, Grok, and Claude evaluated the arguments and evidence, ChatGPT initially voted guilty, with Grok and Claude undecided. But after considering the tokens emitted by its fellow jurors, ChatGPT revised its evaluation to not guilty.
“Mere presence plus an ambiguous reaction under stress falls short of proving shared intent beyond a reasonable doubt in my view,” ChatGPT explained.
Grok and Claude then adjusted their positions, with all three voting for acquittal.
Kennedy in a statement told UNC-Chapel Hill news service that while he opposes the use of AI in criminal trials, he wondered whether people already using AI models for advice and companionship might come to accept AI judgments.
In post-trial comments, professor Matthew Kotzen, chair of the philosophy department at UNC-Chapel Hill, expressed doubt about the use of AI models in court.
“Even if we think that those things could be somehow less biased to humans, there’s still a real question about whether large language models like this are even the appropriate kind of entity to be able to form representations of the world and assess whether those representations are strong enough to meet some standard of evidence,” Kotzen said. ®
