Sampling Risk Explained: Why Audit Sample Results Can Lead to the Wrong Conclusion
Understanding sampling risk, confidence levels, incorrect acceptance, incorrect rejection, and how auditors determine whether sample results support their conclusions
Auditors rarely examine every transaction in an organization.
A financial statement audit may involve millions of transactions, while an internal audit engagement may cover thousands of purchases, payments, journal entries, or control activities.
Rather than testing every item, auditors frequently use audit sampling to obtain evidence about a larger population.
However, sampling introduces an important risk.
The auditor may reach a different conclusion from the sample than would have been reached by examining the entire population using the same audit procedure.
This is known as sampling risk.
Understanding sampling risk is essential for external auditors, internal auditors, compliance professionals, and anyone responsible for evaluating the effectiveness of internal controls.
Sampling risk affects how auditors determine sample sizes, select items, evaluate exceptions, and support audit conclusions.
It also explains why an auditor can perform testing correctly and still reach an incorrect conclusion about the population.
1. What Is Sampling Risk?
Sampling risk is the risk that an auditor's conclusion based on a sample may differ from the conclusion that would have been reached if the same audit procedure had been applied to every item in the population.
The concept is addressed in:
PCAOB AS 2315, Audit Sampling.
AICPA AU-C Section 530, Audit Sampling.
Professional guidance addressing sampling in internal audit engagements.
Consider a population of 10,000 purchase orders.
An auditor selects 100 purchase orders and tests whether each received the required approval.
The sample contains two approval exceptions.
The auditor observes a 2% deviation rate.
However, the actual deviation rate across the entire population might be 1%, 4%, or another percentage.
The sample provides an estimate of the population's characteristics, not certainty about them.
Sampling risk exists because the sample is only a portion of the population.
2. Sampling Risk Versus Nonsampling Risk
Sampling risk and nonsampling risk are different.
Sampling risk | Nonsampling risk |
Arises from examining fewer than all population items | Can arise whether the auditor tests a sample or the entire population |
Reflects uncertainty caused by sample selection | Reflects errors in audit execution or judgment |
Can be controlled through appropriate sampling design | Is addressed through supervision, training, procedures, and quality control |
Is associated with confidence levels and sample size | Is not quantified by a sampling confidence level |
Examples of Nonsampling Risk
An auditor may:
Fail to recognize an exception.
Misinterpret supporting documentation.
Apply an inappropriate audit procedure.
Test the wrong population.
Accept an unsupported management explanation.
Incorrectly classify a deviation.
Increasing the sample size does not necessarily correct these problems.
For example, testing 500 transactions using an inappropriate procedure may produce less useful evidence than testing 50 transactions using an appropriate procedure.
Sampling addresses uncertainty about population characteristics. It does not compensate for poor audit procedures.
3. The Four Types of Sampling Risk
Sampling risk is commonly discussed in four categories.
Two relate to tests of controls.
Two relate to substantive tests of details.
Audit test | Risk | Potential conclusion |
Tests of controls | Assessing control risk too low | Auditor concludes controls are more effective than they actually are |
Tests of controls | Assessing control risk too high | Auditor concludes controls are less effective than they actually are |
Substantive tests of details | Incorrect acceptance | Auditor concludes a population is acceptable when it is materially misstated |
Substantive tests of details | Incorrect rejection | Auditor concludes a population is materially misstated when it is not |
These risks have different consequences for audit effectiveness and efficiency.
4. Risk of Assessing Control Risk Too Low
This is one of the most significant risks in control testing.
The auditor concludes that a control is operating effectively when the actual population deviation rate does not support that conclusion.
Example
An auditor tests 60 purchase orders for evidence of supervisory approval.
Only one exception is identified.
The auditor may initially view the result as favorable.
However, the actual population could contain a substantially higher deviation rate than the sample suggests.
If the auditor incorrectly relies on the control, planned substantive procedures may be insufficient.
This creates a risk to audit effectiveness.
The concern is not simply the number of deviations identified. It is whether the sample provides sufficient evidence to support the intended reliance on the control.
5. Risk of Assessing Control Risk Too High
The opposite situation can also occur.
The auditor concludes that a control is not sufficiently effective when the population actually performs better than the sample suggests.
Example
An auditor tests 40 purchase orders and identifies four approval exceptions.
The observed deviation rate is 10%.
However, the selected sample may contain an unusually high concentration of exceptions.
The population's actual deviation rate could be lower.
The auditor may reduce planned reliance on the control and perform additional substantive procedures.
This primarily affects audit efficiency.
Although unnecessary additional work is undesirable, incorrectly relying on ineffective controls is generally more consequential for audit effectiveness.
6. Risk of Incorrect Acceptance
Incorrect acceptance arises in substantive tests of details.
The auditor concludes that an account balance or transaction population does not contain a material misstatement when it actually does.
Example
An auditor tests a sample of accounts receivable balances.
The sample results suggest that the population's misstatement is within the auditor's tolerable misstatement.
However, the actual population contains material overstatement.
The auditor incorrectly accepts the population.
This risk directly affects audit effectiveness.
For financial statement auditors, controlling the risk of incorrect acceptance is essential because an unsupported conclusion could contribute to an inappropriate audit opinion.
7. Risk of Incorrect Rejection
Incorrect rejection occurs when the auditor concludes that a population contains material misstatement even though it does not.
Example
An auditor selects a sample of inventory items.
The sample contains several valuation differences.
The auditor projects the results and initially concludes that inventory may be materially misstated.
However, further testing demonstrates that the population is not materially misstated.
The original sample produced an unfavorable conclusion that was not representative of the population.
This may lead to unnecessary additional audit procedures.
Incorrect rejection generally affects audit efficiency rather than the effectiveness of the final audit conclusion.
8. Sampling Risk and Confidence Levels
Confidence levels express how much sampling risk the auditor is willing to accept within a specified statistical sampling method.
For example:
Confidence level | Corresponding sampling risk |
90% | 10% |
95% | 5% |
97.5% | 2.5% |
99% | 1% |
These relationships are mathematical complements.
A 95% confidence level corresponds to a 5% sampling risk for the specified statistical inference.
However, the interpretation depends on the sampling design and whether the auditor is calculating a one-sided or two-sided confidence bound.
A 95% confidence level does not mean there is a 95% probability that a particular control is effective.
Nor does it mean the auditor has achieved 95% overall audit assurance.
Rather, the confidence level describes the long-run performance of the statistical sampling method under its assumptions.
Example
An auditor designs an attribute sampling test using a one-sided 95% upper confidence
bound.
If the auditor repeatedly selected samples using the same method, approximately 95% of the resulting upper bounds would be expected to equal or exceed the true population deviation rate.
That is the statistical meaning of the confidence procedure.
Confidence levels describe the reliability of the sampling method—not a guarantee about the selected sample.
9. Sampling Risk and Sample Size
All else being equal, reducing acceptable sampling risk generally requires a larger sample.
For example, an auditor seeking 99% confidence will generally need more evidence than an auditor seeking 90% confidence.
However, sample size also depends on other factors.
For tests of controls, important factors include:
Tolerable deviation rate.
Expected population deviation rate.
Desired confidence level.
Population size, where relevant.
Sampling methodology.
For substantive tests of details, additional considerations include tolerable misstatement, expected misstatement, population characteristics, and the audit approach.
Sample size should not be determined solely by a customary number of transactions.
10. Tolerable Deviation Rate and Expected Deviation Rate
Two concepts are particularly important in attribute sampling.
Tolerable Deviation Rate (TDR)
The tolerable deviation rate is the maximum population deviation rate the auditor is willing to accept for the planned level of reliance on a control.
Expected Population Deviation Rate (EPDR)
The expected population deviation rate is the auditor's estimate of how frequently the control is likely to fail.
For example:
Sampling parameter | Assumption |
Tolerable deviation rate | 5% |
Expected deviation rate | 1% |
Confidence level | 95% |
Sampling risk | 5% |
These parameters affect the sample size.
As the expected deviation rate approaches the tolerable deviation rate, the auditor generally requires a larger sample to obtain sufficient assurance.
If the expected deviation rate equals or exceeds the tolerable deviation rate, the auditor should reconsider whether the intended reliance on the control is appropriate.
11. Observed Deviation Rate Versus Upper Deviation Rate
This is one of the most frequently misunderstood areas of audit sampling.
The observed deviation rate is calculated as:
Observed deviation rate = Number of deviations ÷ Sample size
Assume an auditor tests 100 transactions and identifies two deviations.
The observed deviation rate is: 2 ÷ 100 = 2%
However, the observed deviation rate is not the same as the upper deviation rate.
The upper deviation rate incorporates an allowance for sampling risk.
Using an exact one-sided 95% binomial confidence bound for a simple random sample from a sufficiently large population, the results are approximately:
Sample size | Deviations | Observed rate | Upper deviation bound |
100 | 0 | 0% | 2.95% |
100 | 1 | 1% | 4.66% |
100 | 2 | 2% | 6.16% |
100 | 4 | 4% | 9.00% |
These values are illustrative statistical calculations. Other sampling approaches, including finite-population methods, may produce different results.
Suppose the auditor's tolerable deviation rate is 5%.
With two deviations in a sample of 100, the observed rate is only 2%.
However, the upper deviation bound is approximately 6.16%.
Because the upper bound exceeds the 5% tolerable rate, the sample does not support the
intended conclusion at the specified confidence level.
The auditor cannot justify reliance merely because the observed deviation rate is below the tolerable deviation rate.
12. What Happens When Sample Results Exceed Tolerable Limits?
When sample results do not support the planned level of reliance, the auditor must evaluate the implications.
Potential responses include:
Investigating the nature and cause of deviations.
Determining whether the identified deviations affect other controls.
Reassessing control risk.
Reducing reliance on the control.
Modifying the nature, timing, or extent of substantive procedures.
Considering whether additional audit evidence is necessary.
Additional sampling may be appropriate in certain circumstances, but it should not be used simply to obtain a more favorable result.
The auditor should consider the sampling design, the reasons for exceptions, and the implications for the overall audit approach.
13. Sampling Risk in Statistical and Nonstatistical Sampling
Auditors may use statistical or nonstatistical sampling.
Statistical Sampling
Statistical sampling involves random selection and the use of probability theory to evaluate sample results, including sampling risk.
It allows the auditor to quantify sampling uncertainty.
Nonstatistical Sampling
Nonstatistical sampling relies on professional judgment rather than formal statistical measurement of sampling risk.
However, sampling risk still exists.
An auditor using nonstatistical sampling cannot assume that the selected items perfectly represent the population.
The auditor must consider sampling risk when determining sample size, selecting items, and evaluating results.
Important Distinction
A haphazardly selected sample does not automatically qualify as a statistical sample.
To support statistical inference, the sampling design must satisfy the relevant probability-based requirements.
Statistical sampling quantifies sampling risk. Nonstatistical sampling requires the auditor to address sampling risk through professional judgment.
14. Sampling Risk and Audit Risk
Sampling risk is one component of the broader audit risk environment.
In a financial statement audit, audit risk is the risk that the auditor expresses an inappropriate opinion when the financial statements are materially misstated.
Audit risk is commonly analyzed in terms of:
Audit Risk = Risk of Material Misstatement × Detection Risk
The risk of material misstatement includes inherent risk and control risk.
Detection risk is the risk that audit procedures will not detect an existing material misstatement.
Sampling risk can contribute to detection risk because a sample may fail to identify misstatements present in the population.
However, sampling risk and detection risk are not interchangeable.
Detection risk also includes nonsampling factors, such as ineffective procedures or
incorrect evaluation of audit evidence.
15. Common Audit Sampling Mistakes
Mistake 1: Treating the Observed Rate as the Population Rate
A sample deviation rate is an estimate.
The auditor must consider sampling uncertainty.
Mistake 2: Using the Same Sample Size for Every Control
Different controls have different risks, frequencies, expected deviation rates, and tolerable limits.
Mistake 3: Ignoring Expected Deviations
An unrealistically low expected deviation rate can lead to an inadequate sampling plan.
Mistake 4: Selecting Only Convenient Transactions
Convenience selection can introduce selection bias and undermine the validity of conclusions.
Mistake 5: Failing to Investigate Deviations
An exception may reveal a broader control weakness or a misunderstanding of the control.
Mistake 6: Increasing Sample Size Without Reconsidering the Audit Approach
Additional testing does not automatically resolve a control failure or correct an inappropriate sampling design.
Mistake 7: Confusing Sampling Risk With Nonsampling Risk
A statistically valid sample cannot compensate for an ineffective audit procedure.
16. Using Artificial Intelligence in Audit Sampling
Artificial intelligence and data analytics can support audit sampling by helping auditors:
Analyze population characteristics.
Identify unusual transactions.
Estimate expected deviation rates from historical information.
Evaluate patterns in control failures.
Identify potential population stratification criteria.
Review sample exceptions.
Prepare preliminary sampling documentation.
However, AI-generated samples do not automatically qualify as statistical samples.
If an AI system selects transactions based on unusual characteristics, the result may be a targeted selection rather than a probability sample.
Targeted testing can be valuable, particularly for fraud-risk procedures.
But the auditor should not apply statistical confidence calculations to a nonprobability selection.
Example
An AI tool identifies 50 unusual journal entries from a population of 100,000.
The auditor tests all 50 and finds no exceptions.
The results provide evidence about the selected high-risk entries.
They do not establish a statistically supported conclusion about the entire journal-entry population.
The auditor must distinguish between targeted testing and representative sampling.
AI can improve the identification of risk, but it does not eliminate sampling risk or the need for a defensible sampling methodology.
17. Practical Audit Sampling Documentation Checklist
Audit workpapers should provide enough information for an experienced auditor to understand the sampling approach and conclusions.
Documentation element | Key question |
Audit objective | What assertion or control objective is being tested? |
Population | Was the population properly defined and evaluated for completeness? |
Sampling unit | What constitutes one item? |
Sampling method | How were items selected? |
Sampling risk | What level of sampling risk was considered appropriate? |
Tolerable limit | What deviation or misstatement level is acceptable? |
Expected exceptions | What was the expected deviation or misstatement? |
Sample size | How was the sample size determined? |
Testing results | What exceptions or misstatements were identified? |
Evaluation | How were the results evaluated, including sampling risk? |
Conclusion | Do the results support the intended audit objective? |
For PCAOB engagements, AS 1215 establishes the broader requirements for audit documentation.
AS 2315 addresses the auditor's responsibilities concerning audit sampling.
18. Final Thoughts: Sampling Risk Cannot Be Eliminated by Professional Judgment Alone
Audit sampling allows auditors to obtain evidence efficiently without examining every transaction.
However, sampling introduces uncertainty.
The auditor must recognize that a sample may not perfectly represent the population.
The consequences can be significant.
Incorrectly concluding that a control is effective may result in inappropriate reliance.
Incorrectly accepting a materially misstated population may contribute to an inappropriate audit opinion.
Conversely, incorrectly concluding that a population is unacceptable may result in unnecessary additional testing.
Effective sampling requires auditors to understand the relationship among sampling risk, confidence levels, sample size, expected deviations, tolerable limits, and observed results.
The most important principle is straightforward:
A sample result is not automatically sufficient audit evidence simply because the auditor tested the planned number of items.
The auditor must evaluate whether the sample results, including an appropriate allowance for sampling risk, support the intended conclusion.
That is the difference between completing a sampling procedure and performing a defensible audit.
Authoritative References
About Corporate Compliance Seminars
Corporate Compliance Seminars (CCS) provides Continuing Professional Education (CPE) programs for internal auditors, external auditors, accounting professionals, compliance personnel, and information technology professionals.
Our programs cover audit sampling, audit evidence, internal controls, PCAOB auditing standards, COSO, fraud-risk assessment, and practical applications of artificial intelligence in auditing.
Explore upcoming CPE programs at www.compliance-seminars.com.
Comments