A senior judge has raised concerns about the dangers of using artificial intelligence (AI) in preparing for court proceedings following the discovery of references to fictitious legal cases in several documents.
Mr Justice Sweeting expressed these views while rejecting the appeals of two individuals contesting their extradition to Romania for crimes committed there. During the High Court session, it was revealed that two legal authorities cited by the Crown Prosecution Service (CPS) were fabricated by an AI system and did not have any real existence.
Subsequently, the CPS admitted to the error and issued an apology, clarifying that there was no intention to deceive the court. Investigations were launched to understand how the mistake occurred, with the CPS attributing the incident to a failure in verifying the accuracy of the documents before submission.
Although the CPS acknowledged that AI technology played a role in generating the erroneous references, the ultimate cause was deemed to be human error for not ensuring the authenticity of the sources used in the legal submissions. The judge noted that the issue was identified before the court hearing, thereby not affecting the legal arguments or the court’s decision.
In response, the CPS conducted an internal review, examining 78 other cases associated with the same lawyer, which revealed no similar concerns and indicated a low likelihood of recurrence. The judge accepted the CPS’s apology and assurance that there was no deliberate attempt to mislead.
Mr Justice Sweeting emphasized the potential future increase in AI utilization in legal practices but cautioned against its unchecked deployment, particularly in legal research, highlighting the risks involved.
A CPS spokesperson confirmed that AI is not utilized in legal decision-making or charging determinations, reiterating the organization’s apology for the error in the legal submissions. An internal review was conducted promptly to prevent future occurrences and ensure no other cases were impacted by the mistake, with the judge affirming that the error did not influence the court’s final judgment.
The episode underscores the importance of proper oversight when employing AI in legal activities, as stated by the spokesperson.