This is the third and final article in my series about the Virginia Law Review study, “Read but Not Understood? An Empirical Analysis of Consumer Comprehension in Homeowners Insurance,” by Professors Daniel Schwarcz, Brenda Cude, Kyle Logue, and German Marquez Alcala. The first article, “Insurance Experts Can Misread an Insurance Policy: What a Remarkable Virginia Law Review Study Really Shows,” examined the study and the surprising correction the authors made after publication of their earlier working paper. The second, “Courts Have Long Known Insurance Policies Are Hard to Understand: New Research Shows Why,” considered what the research tells judges and insurance regulators about longstanding rules of insurance policy construction.

This final article is about something much bigger than insurance. Artificial intelligence identified an insurance coverage issue the human researchers initially missed. This result should interest every lawyer and academic because it offers a glimpse of how AI may improve the quality of professional work rather than simply make it faster.

AI Was the Second Reader

As discussed in the first article, the researchers originally classified a one-deck collapse as a clear case of a covered loss. When preparing the final Virginia Law Review article, they ran the vignettes and policy language through several OpenAI models, including OpenAI o3. The AI analysis confirmed their conclusions except for that deck-collapse scenario.

Several advanced AI reasoning models noticed another clause the researchers had not fully appreciated. The authors went back to the policy, reconsidered their analysis, and reversed their coverage determination. To their credit, they openly acknowledged that the co-authors with insurance policy expertise had fallen into essentially the same reading problem experienced by many of the study participants. They reached an initial conclusion and did not fully account for later language inconsistent with it.

This occurrence does not prove that AI understands insurance policies better than insurance professors. It does prove human expertise does not make AI useless. Here, the machine found something knowledgeable humans overlooked, the humans verified it, and the final work became better.

This is how I use AI. I do not need a machine constantly agreeing with me. I want it to point out what I missed, what provision cuts against my conclusion, what opposing counsel will argue, whether I followed every cross-reference, and whether another reasonable interpretation exists. The AI does not become the authority. It becomes another reader—an extraordinarily fast one that can be instructed specifically to attack my preliminary conclusion.

And yes, I use it to ask what good ole’ Steve Badger will argue when we debate and how I can best provide a stinging response.  I need all the help I can get.

What Does This Mean for Legal Scholarship?

The implications for empirical legal scholarship are significant. Researchers often have to classify something before they can measure how subjects respond to it. A policy provision may be coded as covered or excluded, ambiguous or unambiguous, enforceable or unenforceable. If that underlying legal judgment is wrong, the empirical conclusions built upon it can also be distorted.

Peer review, research assistants, editors, and other professors remain essential. But why would scholars not also use sophisticated AI systems as an adversarial check? Ask several models to develop the strongest contrary interpretation. Require them to identify the exact language supporting it. Ask what provisions or authorities the researcher may have overlooked. Then verify everything against the original sources.

That is not surrendering scholarship to machines. It is using machines to make scholarship harder to get wrong. The purpose of serious scholarship should not be to demonstrate that the scholar was right the first time. It should be to get as close to right as possible.

The same reasoning applies to lawyers. I expect AI to make me much faster, but speed alone does not particularly excite me. Producing mediocre work twice as fast is not progress. What matters is whether AI allows me to find more relevant authority, identify weaknesses before opposing counsel does, compare more information, and spend more of my time exercising judgment and developing strategy.

Is AI-Assisted Work Still Authentic Work?

There is a legitimate question about authorship and authenticity. If AI helps with research, challenges an argument, suggests another analysis, or improves a draft, whose work is it?

For me, the answer depends upon responsibility. If a lawyer simply asks a machine for an answer, does not understand it, fails to verify it, and puts his name on it, that lawyer has surrendered an important part of his professional obligation. But if AI helps the lawyer research more broadly, think more deeply, test assumptions, and catch mistakes, the lawyer has not surrendered judgment. He has equipped it with a better tool.

Lawyers have always adopted technologies that changed how we work. We moved from paper digests and Shepard’s books to computerized legal research. We use document databases, electronic discovery, spreadsheets, litigation software, and teams of people to help us produce better work. Authenticity cannot sensibly mean that every useful thought must arise without assistance from anything outside our own brains.

The meaningful questions are whether I understand the work, whether I have verified it, whether I exercised independent judgment, and whether I am prepared to defend the result. If the answer is yes, I consider it my work.

Still, this topic personally bothers me. I am working through drafts of a book in which the topic of authenticity and AI is part of the discussion. We already know plenty of people living pretend lives on social media. AI will expand that so much that I think some people will be acting out their lives in a computerized version of themselves.

The Technology Will Get Better

Some lawyers and academics seem determined to point to every current AI mistake as proof that this technology will never become reliable enough for serious professional work. We should absolutely expose those mistakes and verify AI-generated work. But assuming that today’s limitations define tomorrow’s capabilities seems to me like putting one’s head in the sand and hoping the world will stop changing.

It will not. These systems are going to improve. They will reason across larger collections of information, connect provisions and authorities more effectively, check themselves against primary sources, and complete analytical tasks in minutes that once took talented people hours or days.

I also think it is inevitable that properly trained machines will become better than human experts at certain tasks. Machines do not get tired after the seventieth page, become impatient with the tenth cross-reference, or decide they have already found the answer and stop looking. Human beings will remain responsible for judgment, strategy, ethics, persuasion, empathy, and determining what matters. But the machine may become better at finding everything we ought to consider before exercising that judgment.

Kyle Logue’s University of Michigan interview suggests that the researchers are already looking toward that future. Their planned follow-up study will give some participants access to a large language model and examine whether AI helps consumers understand insurance coverage, what they ask it, and where it succeeds or fails. That is an important next step because the published Virginia Law Review article already suggests AI-powered “smart readers” as one possible way to help consumers navigate complicated contracts.

The question will not remain whether people should use AI. Increasingly, the question will be how to use it responsibly and whether those who learn to use it well will outperform those who refuse to do so.

This episode, buried in an insurance law study, may be an early lesson. The researchers were examining whether ordinary homeowners could understand an insurance policy. Along the way, sophisticated insurance experts missed something. An AI system called attention to it, and the researchers produced better scholarship because they listened and checked.

This seems like a pretty good model for the future. Artificial intelligence should not relieve us of the obligation to think. Properly used, it should make us think more carefully, work faster, and produce better work.

Thought For The Day

“We would like to achieve essentially superhuman productivity.”
—Jensen Huang, Founder and CEO of NVIDIA