Looks Right, But Isn't: Building Your Own AI for Entity Review?

Almost two years ago, I started down the path you may be considering right now: building an AI platform to review borrower entity documents.

Before EntityScan AI was a product, it was just me and the leading AI models. My first prompt was three paragraphs long. I fed in an operating agreement, and what came back was remarkable — organized, professional, confident. It identified the members. It laid out the management structure. It read like something a law firm would charge real money for.

If I had been a lender, I would have believed I was done.

The advantage that changed everything

I had something other builders don't: a way to find out when the report was wrong.

Highland Law's attorneys review dozens of borrower entities every single day, and have for nearly three decades. So instead of admiring the output, I tested it — against completed, attorney-prepared reviews where the correct answers were already established. Real deals, real documents, real conclusions a lender had relied on at closing.

The results were not close. The reports were confident, polished, and wrong in ways that mattered: signing authority attributed to people who didn't hold it, amendments that quietly rewrote management authority missed entirely, conflicts between the operating agreement and the articles of organization sailing by unnoticed — and worst of all, signature blocks that weren't adequate to bind the borrower.

Here is the part I can't emphasize enough: nothing about those reports looked wrong. The errors read exactly as smoothly as the correct conclusions sitting next to them.

How would you know?

That's the question I would put to any bank, credit union, or private lender considering building this in-house.

When your internally built reviewer produces its impressive report, who at your shop is equipped to grade it? The underwriter reading it is relying on it precisely because entity-document analysis isn't their specialty. That's what the report is for. If the analysis is wrong — the wrong manager, a missed consent requirement, an unenforceable signature block — the report doesn't flag it. It reads exactly as well as when it's right.

And lending is the worst possible place to find out the hard way, because the feedback is slow and rare. A bad signing-authority conclusion doesn't blow up at closing. It sits quietly in the file, and it only matters on the day you need to enforce — in a workout, in a default, possibly years from now. Which means a tool that's wrong one time in ten can look flawless for a very long time. "It's been working great" and "nobody has checked" are indistinguishable from the inside.

We only learned how much the models miss because we could measure it. Without ground truth to test against, you can't even calculate your error rate — let alone fix it.

What reliable actually took

That three-paragraph prompt didn't stay three paragraphs. The expert knowledge and edge-case rules underlying our LLC document analysis alone now run more than one hundred pages, and they grew that long one hard-won lesson at a time. Nearly every page exists because a real document from a real deal produced a wrong answer that had to be caught, corrected, and verified before the platform could be trusted with the next one. The models aren't weak. The permutations are endless — entity structures arrive in every configuration commercial lending can produce — and reliability means navigating all of them, not most of them.

What made that possible is the measuring stick: validation against 9,000+ attorney-prepared entity reviews, with dozens more added every single day as Highland Law's practice continues. Every new review — anonymized before it ever crosses the secure firewall between Highland Law and EntityScan AI — tests the platform again, and every miss makes it better. At the original publication of the March article, that number was 6,500. The standard anyone trying to build a comparable solution would have to match isn't just high. It's getting higher every day as the EntityScan platform continues to evolve.

The first prompt — a few paragraphsToday — 100+ pages of expert rules

Build where you can verify

I'm not against building AI tools in-house — we use custom-built AI tools in our business every day, and for plenty of workflows building is the right call. The rule I'd offer is simple: build where you can verify. If your team can look at the output and know whether it's right, build with confidence. Entity review fails that test by definition — verifying the output takes the very expertise the tool exists to supply.

You can build something that looks like an EntityScan report. But the report format was never the hard part. The hard part is the reason to trust it: every conclusion validated against 9,000+ attorney-prepared entity reviews — and counting — refined continuously by attorneys who do this work every day. That's what makes an EntityScan report something you can rely on. Not how it looks. How it holds up.

You can build a lot of things. This isn't one of them — at least not a version worth betting your loan closings on.

Benchmark updated September 13, 2026: EntityScan has now been validated against 9,000+ attorney-prepared entity reviews.

Update: What the EntityScan Assistant Adds

The question of what makes an entity review useful continues after the report arrives. A lender may need help understanding a risk factor, deciding how to address it, or working through the steps required to use different signers. The included EntityScan Assistant knows the report in detail and helps with those follow-up questions, including steps and documents when needed.

That expands the comparison discussed in this article. Producing a dependable report is one substantial undertaking. Providing useful assistance grounded in its findings adds another. EntityScan brings the report and that assistance together, so lending teams can get answers and help taking the next steps.

See the report and EntityScan Assistant in action →

Gary Highland is the Founder and CEO of EntityScan AI and Highland Law, with over 30 years of experience in commercial real estate finance. He previously founded GoDocs, the commercial lending industry's leading loan document automation platform.

For the economics side of this decision — data requirements, development costs, and where building does make sense — see Build vs. Buy: The AI Question Every Lender Will Face This Year.