DocketX

DocketX / How to evaluate a legal AI tool

How to evaluate a legal AI tool

Every legal AI product says it is accurate. Here are the questions that separate a tool that checks from a tool that sounds confident, including where we say no.

Questions worth asking any vendor

• Does it look citations up, or just read carefully? Ask what corpus, how many documents, and as of what date. A vendor who cannot state a count and a date is describing a vibe.
• What happens to a citation it cannot verify? The options are: leave it in silently, flag it, or remove it and say so. Only the third protects a filing.
• Does it check the case name, not just the number? A real citation with an invented case name attached passes a naive existence check.
• Are quotations checked against the source text? And does it distinguish an invented quote from an accurate quote of a superseded rule?
• Is there a record? Can you retrieve, per request, what was searched, what was checked, and what was removed?
• What does it refuse to claim? A tool that will not tell you its limits has not measured them.

Our own answers, including the limits

We hold 615,391 Texas appellate decisions through June 2026, 122,681 Texas statute sections, 45,660 U.S. Supreme Court opinion documents, 370 county local rules and 3,967 court rule sections word for word. Unverifiable citations are removed and disclosed. Every request carries a verification report you can retrieve.

What we do not do: we do not hold federal district or circuit opinions, other states case law (Washington is in progress), treatises, or citator history. Verification proves a citation exists and that its name matches, not that it is still good law or that it supports the point. We say unverified rather than nonexistent when our library may simply not cover it, because those are different claims.

What a firm gets →

Questions

Is a bigger model more accurate for law?

Not reliably. In our benchmarking on blinded real Texas cases, grounding in a real library plus verification changed accuracy far more than moving between frontier models.

Can I use my own model?

Yes. DocketRouter puts the same law layer and citation gate around whichever model you choose, including a private model on hardware you control.

How do you know your own numbers?

They are counted from the live library, not estimated, and the benchmark uses real Texas appellate cases with the source opinion withheld from the model.

Related

Keep reading

Why AI invents case citations

The failure mode the checklist is about.

Read →

What a firm gets

How our answers map to the checklist.

Read →

For education

Classroom and clinic use.

Read →

DocketX products are research and explanation tools, not legal advice and not a law firm. They show you the real rule or opinion behind a statement so you can read it yourself. If you can get a lawyer or free legal aid, do.