From Prompting to RAG: How Audit Staff Should Mature Their Use of AI
Most audit firms are already “using AI.”
The problem is that most are stuck at the weakest level—prompting—and mistakenly think they’re further along than they are.
We are in our CPE events teaching reality clearly: there are three very different ways to improve AI, and only one of them changes the source of truth
If you want AI that holds up under partner review, peer review, and PCAOB inspection, your staff must progress deliberately through all three stages.
Stage 1: Prompting — “Tell the AI What to Do”
What it is
Prompting guides AI behavior in real time using written instructions.
How audit staff actually use it
Drafting audit programs
Summarizing standards
Brainstorming risks
Rewriting workpaper narratives
Creating first-pass memos
Why firms start here
Zero setup
Immediate productivity gains
No IT involvement
Hard truth
Prompting does not use firm data.
It relies entirely on the user’s skill and whatever the model learned from public information.
Audit risk
Inconsistent results across staff
Hallucinated standards citations
Weak defensibility if challenged
Prompting controls how AI responds.It does not control what AI knows.
Stage 2: Fine-Tuning / Custom GPTs — “Make AI Talk Like the Firm”
What it is
Fine-tuning shapes AI’s tone, structure, and behavior using examples, templates, and rules.
How audit firms use it
Standardizing workpaper language
Enforcing firm style and terminology
Producing consistent memos and reports
Training staff on “how we write here”
What changes
Outputs become predictable
Review time drops
Junior staff sound more senior
What doesn’t
Fine-tuning does not use live firm data.
It learns patterns—not policies, not current methodology, not engagement specifics.
Audit risk
Becomes outdated when standards change
Requires retraining
Still vulnerable to wrong answers delivered confidently
Fine-tuning controls how AI behaves.
It still does not control what AI knows.
Stage 3: RAG (Retrieval-Augmented Generation) — “Ground AI in Firm-Approved Truth”
What it is
RAG forces AI to retrieve approved firm documents first, then generate answers from those sources.
This is the inflection point.
What RAG connects to
Firm audit methodology
Quality control manuals
PCAOB standards
Engagement-specific workpapers
Client policies and contracts
What changes
AI answers are traceable
Outputs are engagement-specific
Sources are controlled and auditable
AI stops “making things up”
Best audit uses
Policy guidance
Methodology interpretation
Engagement-specific questions
Consistent answers across teams
Real risk
If your source documents are incomplete, outdated, or sloppy—RAG will faithfully return bad answers faster.
That’s not an AI problem.
That’s a documentation problem.
Why This Progression Matters for Audit Staff
Here’s the blunt reality:
Prompting makes individuals faster
Fine-tuning makes firms consistent
RAG makes AI defensible
Only RAG changes the source of truth.
In an audit environment governed by:
PCAOB standards
Peer review
Litigation risk
Inspection scrutiny
Anything less than controlled source data is a liability.
The Bottom Line for Audit Leaders
If your firm is:
Still relying only on prompting → you’re experimenting
Using custom GPTs without RAG → you’re standardizing language, not knowledge
Using RAG → you’re building inspection-ready AI
Or, as our instructors at CCS states it plainly:
· Prompting controls how AI responds.
· Fine-tuning controls how it behaves.
· RAG controls what it knows.
That’s the difference between AI as a convenience and AI as an audit-grade tool.
John C. Blackshire, Jr. Retired CPA