You Know the AI Terms Now. Here’s Where Firms Still Get Tripped Up.

If you’ve read Chapter 5 of Jared Correia’s Legal Tech-to-English Dictionary 2.0: AI Edition on Above the Law, which CosmoLex is proud to sponsor, you can now hold your own in any conversation about AI slop, retrieval-augmented generation (RAG), semantic search, AI governance, and shadow AI. Jared has done the legal profession a real service: he’s translated the vocabulary vendors throw around into plain English, with enough humor to make it stick.

But here’s the uncomfortable truth the dictionary points toward without saying outright: knowing the term is not the same as being ready for what it describes.

Jared frames it well in his chapter. AI isn’t running unchecked in most businesses; people are. “All business management issues are people problems.” That’s exactly right, and it’s why the gap between AI vocabulary and AI readiness is where small and midsize firms are most exposed right now.

This post is the next step after you know the terms. Four places firms still get tripped up, and what to actually do about each one.

Trip-Up #1: You Can Define “AI Governance,” But Do You Have a Written Policy?

Jared defines AI governance as “the full collection of documentation (including policies, procedures, programs, and materials) that guide the appropriate use of artificial intelligence within a business.” Most managing partners can now nod along with that definition.

Far fewer can produce an actual document when asked.

That gap matters, because as Jared’s example dialogue puts it: if you don’t have an AI usage policy, then everybody’s using it, on their own terms, with their own accounts, and with no record of what client information went where.

What belongs in a small or midsize firm’s AI usage policy (this doesn’t need to be a 40-page manual; one clear document beats an aspirational binder):

1. Approved tools list. Name the specific AI tools staff may use, for which tasks, and under which accounts (firm-managed, never personal).
2. Data classification rules. Spell out what may never be entered into an AI tool: client names, matter details, financial records, privileged communications, anything covered by a protective order. If staff can’t classify it, they shouldn’t paste it.
3. Review requirements. No AI-generated output goes to a client, a court, or opposing counsel without human review by someone qualified to catch errors. Cite-checking is non-negotiable.
4. Disclosure standards. When and how the firm discloses AI use to clients, whether in engagement letters, in billing practices, or when a court rule requires it.
5. An approval path. A named person or committee who evaluates new AI tools, so the answer to “can I use this?” is never “just try it and see.”
6. Consequences and reporting. What happens when the policy is violated, and a no-blame channel for staff to self-report mistakes early, because you want to hear about the problem before the client or the bar does.

Review it quarterly. AI tools change faster than any policy cycle law firms are used to. Here’s the catch with that policy, though: it only governs the AI use you know about, which brings us to the term Jared put at the center of his chapter.

Trip-Up #2: Shadow AI Isn’t Hypothetical. It’s Probably Happening at Your Firm This Week.

Of all the terms in Chapter 5, shadow AI (“unauthorized use of artificial intelligence within a business organization”) is the one to take personally. Jared’s example is played for laughs: a staffer feeding a top client’s most sensitive data into a free consumer AI tool to make a word cloud. But the joke lands because everyone reading it knows a Jameson.

The pattern is consistent: shadow AI doesn’t come from malice. It comes from helpful people with deadlines and a free browser tab. And free consumer tools are exactly where the risk concentrates, since consumer-tier AI products may retain prompts, use them for training, and offer none of the confidentiality controls client data requires.

How to spot shadow AI before it becomes a compliance problem:

• Ask, without the ambush. Run an anonymous survey: “Which AI tools have you used for work tasks in the past 90 days?” Amnesty gets you honesty; audits get you silence.
• Look at the browser data you already have. Firm-managed devices and networks can surface which AI domains are being visited on work time. You’re not surveilling; you’re inventorying.
• Watch for the tells in work product. Uniform paragraph rhythm, confidently wrong citations, and formatting that doesn’t match your templates are all giveaways. Reviewers who know what AI output looks like catch it early.
• Check the expense reports and app approvals. Personal AI subscriptions expensed to the firm, or unapproved integrations connected to firm email and document accounts, are shadow AI with a paper trail.
• Close the loop with a sanctioned alternative. Shadow AI thrives where the approved path is worse than the workaround. If staff is sneaking out to consumer tools, that’s demand. Meet it with a vetted option inside your existing systems.

Offering a sanctioned alternative, though, means choosing one. And the moment you start shopping, you run into the third trip-up: every vendor’s deck now says the same two words.

Trip-Up #3: “RAG-Powered” and “Semantic Search” Are Now Marketing Words, and Firms Can’t Verify the Claims

Jared’s definitions of RAG (LLMs accessing specialized datasets to strengthen their output) and semantic search (surfacing information based on intent and context, not just keywords) describe genuinely useful technology. The problem: those same words now appear on vendor slides whether or not the technology behind them is real, mature, or relevant to legal work.

You don’t need an engineering degree to cut through it. You need better questions.

Questions that separate real capability from AI-washing:

• “What data does the AI actually retrieve from, and can you show me?” A real RAG implementation can tell you exactly what it’s grounded in: your firm’s matters and documents, a curated legal dataset, or something else. “It’s trained on a large corpus” is not an answer.
• “What happens when the answer isn’t in the data?” Good systems say so. Bad ones improvise. Ask the vendor to demo a question the system can’t answer and show you how it responds.
• “Where does our client data go, and is it used to train your models?” Get the answer in writing, in the contract, not in a sales call.
• “Can we see the source for every answer?” If the tool can’t show its work (the document, clause, or record an answer came from), your staff can’t verify it, and unverifiable output creates work instead of saving it.
• “What does this look like in our workflow, on our data?” Insist on a demo with realistic legal scenarios, not the vendor’s polished sandbox.

A vendor who welcomes these questions is telling you something. So is a vendor who doesn’t. But those questions only vet whether the AI is real. The final trip-up is about where the AI lives, and it’s the distinction that determines whether the tool you choose reduces your governance burden or quietly adds to it.

Trip-Up #4: AI Bolted On vs. AI Built In. Firms Don’t Know to Ask Which One They’re Getting.

Bolted-on and built-in AI can look identical in a demo. Under the hood, they behave very differently.

How to tell which one you’re getting, and why it matters:

• Bolted-on AI means more seams. A general-purpose tool dropped alongside (or hastily grafted onto) a legacy system usually means a separate login, a separate data flow to audit, a separate vendor agreement to vet, and one more place client data travels to. Every seam between the AI layer and your system of record is a place where oversight breaks down.
• Badly integrated AI recreates shadow AI inside your official stack. If the AI layer doesn’t share your platform’s permissions and logging, you’ve just re-imported the exact problem your policy was written to solve, this time with a purchase order attached.
• Built-in AI keeps one compliance perimeter. AI that lives inside the practice management platform already holding your matters, documents, billing, and trust accounting is governed by the same permissions, audit trails, and guardrails as everything else. Nothing new to learn, no side channel for client data.
• Ask the question directly. “Is your AI native to the platform, or is it a third-party layer? Does it inherit our user permissions and audit logging, or does it have its own?” The answer tells you how much new governance burden you’re buying along with the features.

For what it’s worth, this is the approach CosmoLex has taken: AI integrated directly into everyday workflows, such as summarizing documents, filtering matters and invoices in plain language, automating intake, and streamlining firm workflows, rather than a separate AI product balanced on top of a legacy system.

The Terms Were Step One

Jared’s dictionary closes the vocabulary gap. Download the full eBook from his Above the Law post; it’s worth having on hand for the next vendor pitch. But vocabulary was always step one.

Step two is the unglamorous part: write the policy, surface the shadow AI, pressure-test the vendor claims, and choose AI that lives inside your compliance perimeter instead of outside it. As Jared says, the best news is that controlled AI usage can be a boon for everyone involved. The firms that get tripped up aren’t the ones that adopted AI too slowly. They’re the ones that adopted it without ever deciding how.

Want to see what built-in AI looks like in practice? Explore CosmoLex’s platform or schedule a demo.