California lawyers who hand a brief to a chatbot may soon have to own every word it produces. The state Legislature has passed SB 574, a bill introduced by State Senator Tom Umberg that spells out what an attorney owes a client and a court when generative AI is part of the job. The measure now sits with Governor Gavin Newsom, who can sign it into law or send it back.
The timing is not accidental. Courts across the United States have spent the past three years dealing with filings that cite cases which do not exist, invented by language models and pasted into motions by lawyers who never checked. California’s answer is not a ban. SB 574 lets attorneys keep using AI for research, drafting and document review. What it does is close the door on blaming the tool when something goes wrong.
What the bill actually requires
Four duties sit at the centre of the text. Lawyers would need to take reasonable steps to verify anything an AI system generates, including checking that legal citations are real and correcting false information before it goes anywhere. They would have to disclose AI use in certain court filings. They would be required to protect confidential and nonpublic client information when using these tools, which in practice limits what can be fed into public AI services. And they could not delegate legal work or judgment to an AI system outright; the attorney remains the decision-maker, whatever drafted the first version.
The bill reaches beyond law firms too. Arbitrators would be barred from handing decision-making power to AI. They could still use software to organise documents or summarise submissions, but the ruling has to come from a person.
Old duties, new packaging
Some legal technology specialists have pointed out that much of SB 574 restates obligations California lawyers already carry. Competence, candour to the court and confidentiality are not new concepts, and the State Bar of California has already issued guidance on generative AI and is reviewing further ethics rules covering professional judgment, source checking and firm-level AI policies. The bill was also modelled in part on a California Judicial Council rule that governs how judges and court staff may use AI.
That criticism misses why the bill matters. Guidance from a bar association is advisory. A statute is not. Writing the duties into law gives judges a clear hook for sanctions and gives clients a clear standard to hold their lawyers to. It also removes the ambiguity that has let some practitioners treat AI output as a black box they are not responsible for.
What changes inside a law firm
If Newsom signs, compliance teams at California firms will have work to do. Expect written policies naming which AI tools are approved, rules on what client data may and may not be entered into them, mandatory citation checks on any AI-assisted filing, a process for the new disclosure requirement, and some form of record-keeping showing how AI was used on a matter. Firms will also need to decide who owns this: a dedicated AI committee, or an extension of existing technology and risk policies.
Training follows. Associates and staff will need to understand where these tools fail, not just where they save time. Recruiters are already starting to look for lawyers who can use AI responsibly, and law schools are likely to fold AI risk into research and writing courses.
There is a commercial upside hiding in the paperwork. Corporate clients increasingly ask how their outside counsel uses AI, and a firm that can point to a documented, statute-compliant process has an easier answer than one that cannot.
Why the rest of the country is watching
California is one of the largest legal markets in the country, and its rules tend to travel. Other states have handled AI in the courtroom through standing orders from individual judges or bar opinions, a patchwork that leaves lawyers guessing from one courtroom to the next. A signed SB 574 would give other legislatures a template with a simple premise: use the tools if you like, but the human signs the filing and the human answers for it.
The bill’s fate now rests on one signature. Newsom has vetoed high-profile AI legislation before, so nothing is settled until the ink is dry. What is clear is that the era of treating AI hallucinations in court as an honest mistake is ending, in California first and probably elsewhere soon after. For more coverage of AI in the legal profession, visit Mylistingo.
