Courts around the world are confronting a problem that legal professionals did not anticipate when they began adopting AI tools: the machines are sometimes making up case law. Entirely. Convincingly. With fictional citations that look indistinguishable from real ones until a clerk or opposing counsel tries to find the original judgment and discovers it does not exist.
A Pattern That Won’t Stop
By December 2025, US federal and state courts had collectively logged more than 660 documented incidents of AI-generated legal filings containing fabricated citations, according to tracking maintained by the legal research platform Fastcase. The rate was running at four to five new cases per day in early 2026 — not slowing down despite the wave of judicial sanctions, professional conduct warnings, and widespread coverage of early high-profile cases.
The incidents follow a consistent pattern. A lawyer, paralegal, or in some cases a self-represented litigant uses a general-purpose AI tool to research a point of law. The tool returns a plausible-sounding case name, a plausible-sounding jurisdiction, a plausible-sounding holding. The person, not knowing to check, cites it in a filing. The other side’s counsel or the court’s clerks search for the case and find nothing. Sanctions, embarrassment, and in some cases bar referrals follow.
The legal profession has a name for this failure mode: hallucination. In the AI context, it refers to outputs that are confident, coherent, and wrong — generated by a model that has no mechanism for distinguishing between a case it has genuinely seen in training data and one it has confabulated to fit the pattern of the question.
The Industry Response
The legal technology sector has moved quickly to position specialised tools as the answer. Westlaw AI, Lexis+ AI, Casetext’s CoCounsel, and a growing field of competitors have built their products on retrieval-augmented generation: before answering a legal question, the system retrieves actual documents from a verified legal database, then uses an AI model to synthesise the answer from those real sources. The citation is to a document the system has actually retrieved, not invented.
The distinction matters enormously in practice. A general-purpose AI tool has no database of verified legal texts to retrieve from — it generates from statistical patterns in its training data. A RAG-based legal tool can show you the source document for every claim. Legal professionals who have used both describe the difference in trust as night and day.
More than half of UK law firms reported in a 2026 survey by the Law Society that they are now using AI tools for document review, contract analysis, and due diligence — making it the most widely adopted application of AI in the sector. The same survey found that only 23 percent had implemented formal policies governing which AI tools could be used for which tasks, suggesting that governance is lagging behind adoption.
Workflow-Native Copilots
The most significant development in legal AI in 2026 is not a standalone tool but an integration model. Rather than requiring lawyers to leave their document management system or case management platform to query an AI, the leading legal AI products are embedding themselves directly into the workflows where legal work actually happens.
Microsoft’s Copilot integration with document management systems now allows a lawyer to right-click a clause in a contract and ask for comparable language from precedent deals — without leaving the document. Specialised legal platforms are building similar functionality into matter management software, time-recording systems, and client portals. The insight driving this shift is that adoption rates for standalone AI tools in law firms are poor; adoption rates for capabilities embedded in tools lawyers already use are dramatically higher.
Courts are also experimenting with AI-assisted case management. Several US district courts are piloting tools that can flag scheduling conflicts, summarise motion history, and draft routine orders for judicial review. The tools are strictly advisory — a judge reviews and signs everything — but the administrative time savings are significant in courts that are managing thousands of active cases simultaneously.
The Human Judgement Question
The AI hallucination problem has prompted a broader debate in the legal profession about what tasks AI should and should not be trusted to perform autonomously. The emerging professional consensus draws a distinction between pattern-matching tasks — finding relevant precedents, checking for defined terms, identifying standard clauses — where AI performs reliably, and judgement tasks — advising on strategy, assessing credibility, predicting outcomes in novel situations — where human expertise remains essential and AI assistance should be treated as input rather than output.
This is not a counsel of caution so much as a description of where the technology actually is. The tools that work best in legal settings are the ones designed with this distinction in mind: surfacing relevant information efficiently, leaving judgement to the lawyer, and generating citations only from verified sources that can be checked.
For more on AI across law, business and society, follow Mylistingo.
