Legal Tales | Why AI cannot – and should not – replace human judges
The limitations of large language models like ChatGPT go well beyond hallucinations, and the courtroom is no place for probabilistic guesswork
Judges, apparently, are next, with some recent research suggesting that machines are more “accurate” in following established legal principles than human judges.
While disruption will surely come, suggesting total replacement is imminent misconstrues what AI systems do and what our legal system and courts exist to achieve.
The starting point is this: insofar as we humans who designed AI can understand them, the large language models behind tools like ChatGPT are, at heart, pattern-completion machines. They ingest vast quantities of internet text and learn to predict, statistically, what word comes next.
Much has already been said about “hallucination”: the tendency of these models to fabricate cases and conjure things out of thin air – a product of the predictive, pattern-completing nature of AI reasoning.
That problem is by now well documented and need not be rehashed at length here. It suffices to say that the issue remains unsolved. But the deeper difficulties are structural, and these deserve closer attention.