13 min
AI at the legal desk
One legal research company promised citations free of made-up answers. Researchers tested it anyway. AI now sorts a lawsuit's documents, drafts contracts and answers legal questions with citations attached. Here's where it helps a company's legal team, where it still makes things up, and what a lawyer checks before signing.
Ticket 01 · The guess
Hallucination-free?
Words to know
Sources and every number (1)
Sources
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Magesh, Surani, Dahl, Suzgun, Manning and Ho; Journal of Empirical Legal Studies, 2025; 202 queries run March to May 2024; abstract and Table B1) · Journal of Empirical Legal Studies (Cornell Law School and Wiley), open access, 2025
Every number
- 100%
doi.org ·2025 · LexisNexis's own 2024 claim for Lexis+ AI's linked legal citations (free of hallucinations), as quoted in the study. - 202 questions
doi.org ·2025 · Legal questions the researchers put to each tool, run between March and May 2024. - 17%
doi.org ·2025 · Share of Lexis+ AI's answers that held a hallucination (a false statement, or a source that doesn't say what the answer claims), the best of the tools tested (the study's table of hallucination rates; the abstract gives seventeen to thirty-three percent for the legal tools). - 33%
doi.org ·2025 · Share of Westlaw AI-Assisted Research's answers that held a hallucination (the study's table of hallucination rates). - 43%
doi.org ·2025 · Share of GPT-4's answers, a general chatbot tested for comparison, that held a hallucination (the study's table of hallucination rates).