Why Most Audit Firms Are Using AI Backwards — and How to Fix It

Audit firms are adopting AI fast.

Audit firms are adopting AI well? Not so much.

Most firms jump straight into using ChatGPT-style tools and assume they’ve “implemented AI.” In reality, they’ve only adopted the least reliable layer of the technology.

Three Ways to Improve AI — Only One Changes the Truth Source

AI in an audit firm evolves in three distinct stages. Skipping steps—or stopping too early—creates real risk.

1. Prompting: Fast Answers, Fragile Results

Prompting is where nearly every firm starts.

Auditors type instructions into an AI tool and get:

• Draft audit programs

• Risk brainstorms

• Workpaper narratives

• Research summaries

It feels productive because it is.

But here’s the problem: prompting does not use firm data.The AI is pulling from general training, not your methodology, your QC policies, or your engagement files.

Result

• Different staff get different answers

• Quality depends on who writes the prompt

• Outputs sound confident—even when they’re wrong

Prompting improves speed, not control.

2. Fine-Tuning & Custom GPTs: Consistency Without Currency

The next step firms take is fine-tuning or building Custom GPTs.

This helps by:

• Standardizing language

• Enforcing firm tone

• Producing consistent memo structures

• Making junior staff sound more senior

That’s valuable—but limited.

Fine-tuned models learn patterns, not live information. They don’t “know” when:

• Standards change

• Firm policies are updated

• Engagement-specific facts matter

Risk

Your AI becomes polished—and outdated.

Fine-tuning controls behavior, not knowledge.

3. RAG: Audit-Grade AI Starts Here

Retrieval-Augmented Generation (RAG) is where AI finally becomes defensible for auditors.

Instead of guessing, the AI:

1. Retrieves approved firm documents

2. Uses only those sources to generate answers

3. Produces responses that are traceable and reviewable

RAG can be connected to:

• Firm audit methodology

• QC manuals

• PCAOB standards

• Engagement workpapers

• Client policies and contracts

Now the AI isn’t relying on “what it remembers.”It’s relying on what you approved.

This is the first time AI has a controlled source of truth.

Why This Matters for Audit Firms

Inspection and Peer Review risk doesn’t come from using AI.It comes from using AI without controls.

• Prompting = convenience

• Fine-tuning = consistency

• RAG = defensibility

If your firm can’t explain:

• Where the AI got its answer

• Which document it relied on

• Whether the source was current and approved

You don’t have an AI strategy—you have an exposure.

The Maturity Path Audit Firms Should Follow

Smart firms don’t abandon prompting or fine-tuning.They layer them correctly.

• Prompting trains auditors to think clearly

• Fine-tuning standardizes firm communication

• RAG anchors AI to firm-approved truth

That progression mirrors how audits already work:judgment → standards → evidence.

Final Thought

AI will absolutely make auditors faster.But speed without control is how audit failures happen.

If your firm wants AI that survives partner review, peer review, and PCAOB inspection, the goal isn’t “better prompts.”

The goal is controlling what the AI knows.

That’s what RAG delivers.

“Modern Auditors Document Better, Think Smarter, and Use AI Responsibly.”

John C. Blackshire, Jr., Retired CPA