Testing the Operating Effectiveness of Internal Controls: Proving That Controls Actually Work

A Practical CPE Program for Internal Auditors, SOX Professionals, Compliance Teams and Internal Control Specialists

A control can be perfectly designed and still fail in practice.

That is why internal control testing cannot stop with design effectiveness.

Once management and auditors determine that a control is capable of addressing an identified risk, the next question is more demanding:

Did the control actually operate as designed, throughout the period, by people with the authority and competence to perform it effectively?

That is the essence of operating effectiveness.

Corporate Compliance Seminars’ Testing the Operating Effectiveness of Internal Controls CPE program is designed to help auditors and control professionals answer that question using practical testing techniques for entity-level controls, financial transaction controls and IT general controls. The course also addresses COSO, SOX compliance, reporting deficiencies, control maturity and continuous monitoring.

For organizations that have already invested heavily in documenting controls, operating-effectiveness testing is where those control descriptions are put to the test.

Design Effectiveness Is Only the Beginning

Internal control evaluation generally involves two separate conclusions.

Design effectiveness asks:

If the control operates exactly as intended, is it capable of preventing or detecting the identified risk?

Operating effectiveness asks:

Did that appropriately designed control actually operate as intended?

PCAOB Auditing Standard 2201 makes this distinction explicit. The standard states that operating-effectiveness testing determines whether the control is operating as designed and whether the individual performing it possesses the necessary authority and competence.

A control can therefore be:

• Well designed and operating effectively

• Well designed but operating ineffectively

• Poorly designed but performed consistently

• Missing entirely

Only the first condition provides the assurance management expects from an effective control environment.

A Control Narrative Does Not Prove the Control Worked

Consider this control:

“The Controller reviews the monthly bank reconciliation.”

The control appears reasonable.

The auditor still needs to determine:

• Was the reconciliation actually prepared every month?

• Was it prepared on time?

• Did the Controller actually perform the review?

• What did the Controller examine?

• Were reconciling items investigated?

• Were unusual items resolved?

• Was evidence of review retained?

• Did the Controller possess appropriate competence?

• Did the control operate throughout the period?

The policy may be excellent.

The narrative may be accurate.

The control may even have operated correctly during the walkthrough.

None of those facts proves that the control operated effectively for the period being tested.

Operating-effectiveness testing requires evidence.

Inquiry Alone Is Not Enough

One of the most important principles in control testing is that asking management whether a control operated does not establish operating effectiveness.

PCAOB Auditing Standard 2201 identifies four common procedures used to test controls:

• Inquiry

• Observation

• Inspection of relevant documentation

• Reperformance

The standard also ranks those procedures generally from less persuasive to more persuasive evidence and specifically states that inquiry alone does not provide sufficient evidence to support a conclusion about control effectiveness.

That is a critical lesson for Internal Audit and SOX teams.

Management saying:

“Yes, we perform the review every month.”

is useful information.

It is not the conclusion.

The auditor should corroborate the statement.

Use a Mix of Testing Procedures

Strong operating-effectiveness testing ordinarily combines multiple procedures.

Inquiry

Ask the control owner to explain:

• What the control does

• How frequently it operates

• What information is used

• What constitutes an exception

• What happens when an exception is identified

Inquiry provides context.

Observation

Watch the individual perform the control.

Observation can help determine whether:

• The described procedure matches actual practice

• The employee understands the control

• The control relies on undocumented workarounds

• The process differs from the narrative

Observation is powerful but usually provides evidence only for the moment observed.

Inspection

Examine documentation showing that the control operated.

Examples include:

• Signed reconciliations

• Electronic approvals

• Workflow records

• Review notes

• Exception reports

• Access reviews

• System logs

Inspection allows the auditor to test activity across a period.

Reperformance

Independently execute the control or selected elements of it.

For example:

• Recalculate the reconciliation

• Reperform a three-way match

• Recreate an access review

• Recalculate a management-review threshold

Reperformance can provide particularly persuasive evidence.

Test the Control Over the Right Period

Timing matters.

A control tested successfully in January may not have operated effectively through December.

PCAOB Auditing Standard 2201 notes that testing controls over a longer period generally provides more evidence than testing over a shorter period. Testing closer to management’s assessment date may also provide stronger evidence about the condition of controls near year-end.

The auditor therefore needs to balance:

• When testing occurs

• How long the control has operated

• Whether the control changed during the year

• Whether additional roll-forward procedures are required

• Whether the control operates daily, weekly, monthly, quarterly or annually

A quarterly control should not be evaluated using the same testing approach as a control performed thousands of times per year.

Sample Size Should Follow Risk and Control Frequency

Auditors often ask:

“How many items should I test?”

There is no universal number.

Sample size depends on factors including:

• Frequency of control operation

• Risk associated with the control

• Expected deviation rate

• Tolerable deviation

• Reliance being placed on the control

• Nature of the evidence

• Population size

• Audit methodology

A control performed once annually may require a different approach from a daily control.

The objective is not to test an arbitrary number of items.

It is to obtain sufficient appropriate evidence to support the conclusion.

Test the Person as Well as the Procedure

Operating effectiveness depends partly on who performs the control.

PCAOB Auditing Standard 2201 specifically requires consideration of whether the person performing the control possesses the necessary authority and competence.

The auditor should therefore ask:

• Does the control owner understand the risk?

• Does the person possess the technical competence?

• Does the individual have sufficient authority to challenge exceptions?

• Can management override the reviewer?

• Has responsibility changed during the year?

• Was the control delegated during absences?

• Did substitutes have appropriate competence?

A review is not effective merely because someone signed the document.

The reviewer must understand what they are reviewing.

Management Review Controls Require Special Attention

Management review controls are common in SOX programs.

Examples include:

• Budget-to-actual reviews

• Financial statement reviews

• Variance analysis

• Reserve reviews

• Forecast reviews

• Reconciliation reviews

• KPI monitoring

These controls can be powerful.

They can also be difficult to test.

The auditor should determine:

• What data was reviewed?

• Was the information complete and accurate?

• What threshold caused management to investigate?

• What level of precision did the review achieve?

• Which unusual items were identified?

• What follow-up occurred?

• What documentation remains?

A manager initialing a spreadsheet may prove that the file was opened.

It does not necessarily prove that a sufficiently precise review occurred.

Information Produced by the Entity Must Be Reliable

Many controls depend on reports generated by company systems.

Examples include:

• Aging reports

• Purchase-order exception reports

• User-access listings

• Journal-entry reports

• Inventory reports

• Duplicate-payment reports

• Budget variance reports

If the information used by the control is unreliable, the control itself may also be unreliable.

Auditors should understand:

• Where the data originates

• Whether the report population is complete

• Whether report parameters are correct

• Whether users can manipulate the data

• Whether the report logic changed

• Whether relevant IT general controls support the system

Operating-effectiveness testing therefore frequently requires testing not only what management did, but also the information management used.

Entity-Level Controls Need Evidence Too

Entity-level controls are sometimes tested too casually because they do not always leave transaction-level documentation.

Examples include:

• Tone at the top

• Governance

• Audit Committee oversight

• Management accountability

• Risk assessment

• Ethics programs

• Whistleblower processes

• Monitoring

CCS’s program addresses testing entity-level controls as part of operating-effectiveness evaluation.

Evidence may include:

• Board minutes

• Audit Committee materials

• Risk assessments

• Ethics investigations

• Hotline statistics

• Management certifications

• Corrective-action tracking

• Employee surveys

• Performance evaluations

• Internal Audit reports

Testing entity-level controls requires judgment.

The absence of a traditional sample does not eliminate the requirement for evidence.

Financial Transaction Controls Require Consistent Execution

Transaction-level controls often include:

• Purchase approvals

• Three-way matching

• Vendor validation

• Reconciliations

• Journal-entry approvals

• Credit approvals

• Payment authorization

• Segregation of duties

Operating-effectiveness testing should determine whether those controls worked consistently.

Suppose a control requires all invoices above $25,000 to receive two approvals.

The auditor selects 30 transactions and finds three lacking the second approval.

The analysis should not stop with:

“Three exceptions noted.”

The auditor should determine:

• Why did the exceptions occur?

• Were they concentrated in one department?

• Did one employee cause them?

• Did management override the process?

• Were the transactions otherwise valid?

• Does the deviation indicate a broader control deficiency?

• Should testing be expanded?

Exceptions are evidence.

They need interpretation.

One Exception Does Not Automatically Mean the Control Failed

PCAOB Auditing Standard 2201 recognizes that effective internal control does not provide absolute assurance and that an individual control does not necessarily have to operate without any deviation to remain effective.

That means auditors must exercise judgment.

An exception should be evaluated based on:

• Nature

• Frequency

• Cause

• Risk

• Potential magnitude

• Compensating controls

• Whether it is isolated or systemic

A missing signature may represent a documentation failure.

A missing approval on a high-risk transaction may indicate something considerably more serious.

The auditor needs to understand the difference.

Root Cause Matters

An effective recommendation should address why the control failed.

Common causes include:

• Poor training

• Staffing shortages

• Unclear responsibility

• System limitations

• Weak supervision

• Management override

• Excessive workload

• Inappropriate control design

• Poor monitoring

If ten employees failed to perform a control, the solution may not be:

“Remind employees to follow the procedure.”

The real problem may be that the control cannot reasonably be performed at the required frequency.

Operating-effectiveness testing therefore should feed directly into root-cause analysis.

IT General Controls Can Undermine Everything Else

Modern controls rely heavily on information systems.

CCS’s program includes testing IT general controls and system safeguards as a major component.

Relevant areas include:

• User access

• Privileged access

• Password management

• Change management

• System development

• Data interfaces

• Backup and recovery

• Security monitoring

If inappropriate users can modify system configurations, automated controls may not be reliable.

If reports can be altered without review, management-review controls may be weakened.

Business process controls and IT controls therefore cannot always be evaluated separately.

Continuous Monitoring Can Improve Control Maturity

Traditional control testing often occurs annually.

Modern organizations increasingly use continuous monitoring.

Examples include:

• Automated exception reporting

• Continuous segregation-of-duties analysis

• Duplicate-payment detection

• Privileged-access monitoring

• Automated reconciliation

• Continuous transaction analytics

COSO emphasizes that effective internal controls support more than compliance; they help organizations operate with greater confidence and integrity in their information.

CCS’s course therefore includes control maturity and continuous monitoring as methods for strengthening the organization beyond periodic testing.

AI Can Strengthen Operating-Effectiveness Testing

Artificial intelligence is creating new opportunities for control testers.

Using approved tools, auditors may be able to use AI to:

• Analyze complete transaction populations

• Identify unusual exceptions

• Compare supporting documentation

• Summarize testing results

• Detect patterns across exceptions

• Generate follow-up questions

• Review evidence for inconsistencies

• Assist with workpaper drafting

But AI should not make the final control conclusion.

The auditor still must determine:

• Whether evidence is reliable

• Whether exceptions matter

• Whether additional testing is necessary

• Whether the control operated effectively

AI can accelerate analysis.

Professional judgment remains the auditor’s responsibility.

Better Workpapers Produce Better Conclusions

Operating-effectiveness workpapers should tell the story of the testing.

A good workpaper identifies:

• Control objective

• Risk

• Control description

• Control frequency

• Control owner

• Population

• Testing methodology

• Sample selected

• Evidence examined

• Exceptions

• Follow-up procedures

• Final conclusion

The workpaper should allow an experienced reviewer to understand exactly why the auditor concluded that the control was—or was not—operating effectively.

“Tested with no exceptions” is not enough.

The documentation should demonstrate the work.

Reporting Deficiencies Requires More Than Listing Exceptions

CCS’s program also focuses on developing actionable recommendations and communicating findings to management.

A strong finding should explain:

• Condition — What happened?

• Criteria — What should have happened?

• Cause — Why did it happen?

• Consequence — Why does it matter?

• Corrective Action — What should management do?

The auditor should avoid overwhelming management with technical testing terminology.

Leadership needs to understand:

• What failed

• Why it failed

• What risk exists

• What must change

That is how testing produces organizational value.

Testing Controls Should Improve the Organization

The purpose of operating-effectiveness testing is not merely to satisfy:

• SOX

• External auditors

• Internal Audit

• Regulators

It should help management understand whether its control system actually works.

Effective controls can help organizations:

• Reduce fraud

• Improve financial reporting

• Strengthen compliance

• Protect data

• Improve decision making

• Reduce operational errors

• Increase resilience

Internal control testing should therefore be viewed as an improvement process—not simply a compliance exercise.

Who Should Attend?

Corporate Compliance Seminars designed Testing the Operating Effectiveness of Internal Controls for professionals involved in control testing, governance and risk management, including:

• Internal Auditors

• Compliance Officers

• Risk Managers

• Financial Professionals

• SOX professionals

• Internal control specialists

The course is offered at the Basic level with no prerequisites or advance preparation and provides 4 NASBA-approved CPE credits in Auditing.

What Participants Will Learn

Participants will learn how to:

• Distinguish design effectiveness from operating effectiveness

• Test entity-level controls

• Test financial transaction controls

• Evaluate IT general controls

• Identify and assess control deviations

• Use evidence appropriately

• Develop stronger recommendations

• Improve control maturity

• Use technology and continuous monitoring to strengthen testing

The focus is practical: participants should be able to take the methodology back to their organizations and improve their control-testing programs immediately.

The Bottom Line

A control does not become effective because:

• It appears in a policy

• It appears in a risk-control matrix

• Management says it works

• It passed last year

• Someone signed a document

Operating effectiveness must be demonstrated.

The auditor needs to determine whether:

• The control operated as designed

• It operated throughout the required period

• The control owner possessed appropriate competence and authority

• The evidence supports performance

• Exceptions were identified and evaluated

• The information used by the control was reliable

• The control continues to address the underlying risk

That is the difference between having controls and having effective controls.

Corporate Compliance Seminars’ Testing the Operating Effectiveness of Internal Controls program gives auditors and control professionals a structured way to make that distinction and improve the quality of their SOX, COSO, compliance and internal audit programs.