Risk analysis and statistical sampling in audit - Methodology

October 2003--June 2004

Risk Analysis and Statistical Sampling in Audit - Methodology

-Ms.Parama Sen

1. The risk model

Making an audit assertion with absolute certainty would be vastly expensive. There would always be some risk that audit fails to discover all material errors, even when 100% of the transactions are audited. Recognising this, the auditor defines an audit risk that he is willing to accept or conversely the assurance that he desires to provide that his audit assertions/ opinions are correct. This risk (or assurance) is usually defined as a matter of SAI policy. Using this assurance as input, it is possible to define a sample, using statistical sampling methods, on which audit tests that are carried out give results that can be projected to the entire population. This approach prescribes a uniform audit scrutiny for all transactions in the population. However, all transactions are not equally risky and treating them as such will mean higher costs of audit in less risky transactions on the one hand and the threat that risky transactions will not be detected on the other.

The risk model is an analytical tool for planning and execution. This approach detects high-risk areas where audit effort can be concentrated. Audit can thus focus on areas which are likely to generate better assurance instead of sampling and testing of larger but low risk areas. It structures the audit procedures and re-organises the audit work in terms of risk perception.

The Risk Model can be expressed by the following equation:

OAR = IR x CR x DR

Where, OAR is the overall audit risk acceptable to the auditor

IR is the inherent risk, i.e. the risk that an error will occur in the first place

CR is the control risk, i.e. the risk that internal controls will fail to detect the error

DR is the detection risk, i.e. the risk that the audit procedures will fail to detect the error

And the underlying assumption is that the inpidual risks, viz., IR, CR, DR are independent of each other.

The overall audit risk is defined by the audit institution and hence is a constant pre-determined quantity. The objective for the auditor is to first assess inherent and control risks in the entity, and then to design and perform appropriate compliance and substantive procedures that provide sufficient assurance such that the product of the risks identified is less than or equal to the overall audit risk that the auditor is willing to accept. If the inherent risk and control risk are low, audit will be required to provide less assurance from substantive tests, while if the inherent risk and control risk are high, the amount of assurance required from substantive audit tests will be high.

In the risk model, thus, the auditor assesses the inherent risk and control risk and solves the equation for detection risk. The detection risk (DR) is actually a combination of two risks; analytical procedures risk (AP) which is the risk that analytical procedures will fail to detect material errors and tests of detail risk (TD) which is the risk that detailed test procedures will fail to detect the material errors. These two risks are again considered independent and thus a multiplicative model is possible.

DR = AP X TD

OAR = IR x CR x AP x TD

The auditors exercise professional judgement in assessing the IR, CR and AP. Then solve the model to arrive at the test of details risk(TD).

2. Materiality and audit risk

While risk is concerned with the likelihood of error, materiality deals with the extent to which we can tolerate error. Materiality relates to the maximum possible mis-statements/ error. The auditor needs to do just enough work to conclude that the maximum possible mis-statement/ error at the desired level of assurance is less than the materiality. Materiality is determined from the user’s point of view, and is independent of the overall audit assurance (risk). While making materiality judgements three main factors are considered; the value of the error, the nature of the error and context in which the transaction has occurred. It is normally sufficient to determine a single materiality level for the audit. However, in some situations it may be desirable to use different materiality levels for different components/ areas of audit.

The auditor is concerned only with material errors. Risk assessment will thus focus on the likelihood of material error. To use the risk model, the auditor has thus to specify the materiality level along with the overall assurance required form the audit.

3. To assess inherent risk

Inherent risk assesses the nature, complexity, and volume of the activities that gives rise to the possibility of error occurring in the first place. The assessment of inherent risk factors would to a large extent be based on the knowledge and understanding of the business of the auditee based on our experience from previous audits and identification of events, transactions and practices which may have a significant impact on the audit area.

The major factors that can be considered for assessment of inherent risk in a financial (certification) audit are listed in Annexure A. Different audits will have a different set of risk parameters for assessment of inherent risk.

Inherent risk has to be assessed for each audit assertion/ opinion. Inherent risk factors impacting the audit assertion need to be documented. The risk associated with each inpidual factor is then assessed as high, moderate or low. The assessment is then consolidated for overall assessment of inherent risk. It is possible to assign numerical values to the risk assessed, or the assessment can be done quantitatively in terms of high, moderate and low.

4. To assess control risk

Control risk assesses the adequacy of the policies and procedures in the auditee organisation for detecting material error for identified functions or activities. For assessing the control risk, the auditor considers both the control environment and control systems together. Techniques used to evaluate internal control are narrative descriptions, questionnaires, check lists, flow charts, inspection, inquiries, observation and re-performance of internal controls. The factors that can be considered for assessment of control environment and control systems in a financial (certification) audit are listed in Annexure B. Different kinds of audit will have a different set of control factors to be considered.

The auditor evaluates the control environment and systems (both manual and IT) and places reliance on them. This evaluation is the preliminary systems examinations and are designed to assess whether the activities undertaken by the audited body are in accordance with the statutory and other authorities, whether the audited body’s structure is likely to ensure adequate internal control, the adequacy of general financial controls, whether the employees in areas critical to internal controls are competent and whether there are adequate other general controls in areas relevant to audit. The control risk is then assessed and expressed either in numerical (percentage terms) or qualitative (high, medium, low) terms.

5. To assess detection risk

Having assessed the inherent and control risks, the risk equation can be solved for detection risk, i.e. the assurance required from audit procedures. An assurance guide is placed at annexure C where the required assurance from substantive audit tests can be read off. This assurance level will be used as input in determining the sample size on which the audit tests need to be performed to arrive at the required overall assistance.

6. Risk assessment leads to a stratification of the audit population

Based on the level of assurance required from audit testing of an area and the materiality of errors associated, audit processes are defined. A high likelihood of error in an audit area which requires a high level of assurance of the audit test along with a high significance would, for example make the area a critical concern for audit and one may decide to conduct a 100% check on these kind of areas. Based on the perception of risk and the materiality along with the value of the set of transactions the population is stratified. Each strata of the population will involve a different level of substantive audit checks. The high risk, high materiality items will be subjected to a higher level of substantive audit test, while an area with lower materiality may be tested through analytical methods or test of controls and lesser substantive tests.

As a rule it is prudent to examine all transactions that are inpidually material. The conclusions which can be drawn from a test of items selected on a high value basis will only relate to these items and provide better assurance to the auditor. Similarly, there could be key items which are especially prone to error or other risks, or merit special attention. The auditor may wish to examine these items 100% when forming an audit opinion.

7. Statistical sampling

Sampling means testing less than 100% of the items in the population for some characteristic and then drawing a conclusion about that characteristic for the entire population. Traditionally, auditors use ‘test check’ (or judgmental sampling, non-statistical sampling) approach. This means checking a pre-determined proportion of the transactions on the basis of the auditor’s judgement. This sampling technique can be effective if properly designed. However, it does not have the ability to measure sampling risk and thus audit conclusions reached becomes rather difficult to defend.

For statistical sampling techniques, there is a measurable relationship between the size of the sample and the degree of risk. Statistical sampling procedure uses the laws of probability and provides a measurable degree of sampling risk. Accepting this level of risk, (or conversely at a definite assurance level) the auditor can state his conclusions for the entire population. In sum, statistical sampling provides greater objectivity in the sample selection and in the audit conclusion.

The basic hypotheses of statistical sampling theory are:

  1. The population is a homogeneous group.
  2. There is no bias in the selection of items of the sample. All items of the population have equal chance of being selected in the sample.

8. Attributes and Variable sampling

Statistical sampling may be used in different auditing situations. The auditor may wish to estimate how many departures have occurred from the prescribed procedures; or estimate a quantity, eg., the value (amount) of errors in the population. Based on whether the audit objective is to determine a qualitative characteristic or a quantitative estimate of the population, the sampling is called an attribute or variable sampling.

Attributes sampling estimates the proportion of items in a population having a certain attribute or characteristic. In an audit situation, attribute sampling would estimate the existence or otherwise of an error. Attribute sampling would be used when drawing assurance that prescribed procedures are being followed properly. For example, attribute sampling may be used to derive assurance that procedures for classification of vouchers have been followed properly. Here, the auditor estimates through attribute sampling the percentage of error (vouchers that have been mis-classified) and sets an upper limit of error that he is willing to accept and still be assured that the systems are in place.

Variables sampling estimates a quantity, eg., amount of sundry debtors shown in the balance sheet or the underassessment in a tax circle. Variables sampling has certain drawbacks which can be overcome through monetary unit sampling, which is an attribute sampling which provides quantitative results and is suited to most audit situations.

9. Sampling methods

There are different ways in which a statistical sample can be selected. A simple random sampling ensures that every member of the population has an equal chance of selection. Though simple to administer, the underlying assumption is that the population is homogeneous. In cases where the population is non-homogeneous, a stratified sampling would be a better option. Here the population is sub-pided into homogeneous groups and then a random sampling is done on the groups, ensuring a better representative sample. Each sampling method has its practical use and limitation. The auditor uses his judgement in determining which kind of sampling is best suited to his audit job.

10. Designing a sample

Once the method of sampling is decided, it is essential to design the actual sample. The basic stages that are involved in attributes sampling are mentioned below:

  1. Determining the sample size
  2. Selecting the sample and performing substantive audit tests on the sample
  3. Projecting the results

(a) Determining the sample size:

The first step is to define clearly the target population and the error/ exception (attribute) that audit wishes to test.

The tolerable error or the maximum errors that the auditor is willing to accept and still conclude that the auditee is following the procedures properly.

Audit test on the sample will throw up an estimate of error for the population. The true error of the population could be more than this estimate. The difference between the sample estimate and the actual population is the precision level. The auditor has to decide the precision he desires to provide in his estimates. Tolerable error being the maximum error that the auditor is willing to accept is Maximum (sample estimate + precision level) that is acceptable.

The confidence level or the level of assurance that audit needs to provide is to be defined. When a risk assessment has preceded the sampling process, the confidence level would be (1- detection risk). Confidence level states how certain the auditor is, that the actual population measure is within the sample estimate and its associated precision level.

The occurrence rate or population proportion which is the proportion of items in the population having the error/ exception that audit wishes to test.

The required sample size can be calculated using the formula (annexure D), or read off from standard statistical tables (annexure E) at the required confidence level.

The sample size would be larger, higher the confidence level and precision required. Also if the occurrence rate in the population becomes larger the size of the sample would increase. In case of variables sampling, where the estimate of a quantity is required, sample size becomes a function of the standard deviation in the population rather than the occurrence rate.

(b) Selecting the sample and performing substantive audit tests on the sample

There are a large number of methods of sample selection. The most frequently used method is random selection where each item in the population has a equal chance of selection. This could be done by using random number tables or through computers. In a systematic selection, one or two items are selected randomly, but the other items are selected by adding the average sampling interval. The greatest advantage of this method is that when it is used in monetary unit sampling, it automatically ensures that all items greater than the average sampling interval are selected. However, this method cannot be used when some fixed numbers are assigned to various categories of transactions, which make up the accounts, as either all items of a particular category will be selected or ignored completely. In the cell sampling method, the population is pided into a number of cells and one item is selected from each cell randomly. This method overcomes the drawback of systematic sampling when fixed numbers are given to various categories, but retains the advantage of systematic sampling of automatically selecting items bigger than the average sampling interval.

Auditing software, eg., IDEA is an efficient tool for sample selection. Once the sample is selected, identified audit tests are to be applied on the sample.

(c ) Projecting the results

Once the audit tests are performed on the sample, the test results need to be projected to the population. Following this, a conclusion has to be reached whether the auditor can place an assurance on the systems.

After the audit tests, the auditor obtains the actual number of errors in the sample selected. As the sample size and the confidence level desired by the auditor are known elements, the formula given at annexure D can be used to solve for the precision. The maximum error estimate of the population would then be obtained after loading the sample estimate with the precision. This is the computed tolerable error. Instead of solving the mathematical formula, it is possible to read off the ‘computed tolerable error’ straightaway from the statistical tables for the desired confidence (assurance levels). A sample of such a statistical table is placed at annexure F.

In a case when the computed tolerable error is less than the tolerable error, the auditor can place the desired assurance on the systems. When the computed tolerable error is higher than the tolerable error, the auditor cannot derive assurance from the systems. The auditor may, in such situations reduce the assurance he derives from the control and increase the assurance required from substantive tests.

AnnexureA

Factors to consider for assessment of inherent risk in financial audit

The number and significance of audit adjustments and differences waived during the audits of previous years

Complexity of underlying calculations of accounting principles

The susceptibility of the asset to material fraud or misappropriation

Experience and competence of accounting personnel responsible for the component

Judgement involved in determining amount

Mix and size of items subject to the audit test

The degree to which the financial circumstances of the entity may motivate its management to mis-state the component in regard to this assertion

Integrity and behaviour of the management

Management turnover and reputation

AnnexureB

Factors to consider for assessment of control risk in financial audit

Evaluate the control environment

Management philosophy and operating style

The functioning of the board of directors and its committees, particularly the audit committee

Organisational structure

Methods of assigning authority and responsibility

Management control methods

Systems development methodology

Personnel policies and practices

Management reaction to external influences

Internal audit

Evaluate the control systems

Segregation of incompatible functions

Controls to ensure completeness of transactions being recorded

Controls to ensure that transactions are authorised

Third party controls (e.g. confirmation of events)

Controls over accounting systems

Controls over computer processing

Restricted access to assets( only allow access to authorised personnel)

Periodic count and comparison (ensure book amounts reconcile with actual inventory counts)

Controls over computer operations

AnnexureC

Assurance Guide

Assurance from inherent risk evaluation Assurance from internal control (SBA) Assurance from substantive analytical review procedures Required assurance from detailed substantive tests confidence level
High High
(Excellent system)
Med
Low
Nil
60
70
75
Med
(Good system)
Med
Low
Nil
65
75
80
Low
(Fair system)
Med
Low
Nil
75
80
85
Nil
(Poor System/DST)
Med
Low
Nil
92
94
95
Medium High
(Excellent system)
Med
Low
Nil
75
80
85
Med
(Good system)
Med
Low
Nil
80
85
90
Low
(Fair system)
Med
Low
Nil
85
90
92
Nil
(Poor System/DST)
Med
Low
Nil
95
96
97
Low High
(Excellent system)
Med
Low
Nil
90
92
94
Med
(Good system)
Med
Low
Nil
92
94
95
Low
(Fair system)
Med
Low
Nil
94
95
96
Nil
(Poor System/DST)
Med
Low
Nil
98
99
99

NB Nil assurance from inherent risk evaluation would imply that exception audit procedures would be necessary.

AnnexureD

To calculate sample size for attribute sampling

Sample size (n)=Z 2 p(1-p) ,

E2

Where,Z = score associated with confidence level

E = precision

And p = proportion (occurrence rate in the population)

Z score values:

Confidence level Z score values
80 % 1.28
85 % 1.44
90 % 1.65
95 % 1.96
99 % 2.58

Annexur eE

Statistical Sample sizes for confidence level 95 % with number of expected errors in paranthesis

Occurrence Rate Tolerance Rate
2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10 % 15 % 20 %
0.00% 149(0) 99(0) 74(0) 59(0) 49(0) 42(0) 36(0) 32(0) 29(0) 19(0) 14(0)
.25 236(1) 157(1) 117(1) 93(1) 78(1) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
.50 * 157(1) 117(1) 93(1) 78(1) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
.75 * 208(2) 117(1) 93(1) 78(1) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
1.00 * * 156(2) 93(1) 78(1) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
1.25 * * 156(2) 124(2) 78(1) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
1.50 * * 192(3) 124(2) 103(2) 66(1) 58(1) 51(1) 46(1) 30(1) 22(1)
1.75 * * 227(4) 153(3) 103(2) 88(2) 77(2) 51(1) 46(1) 30(1) 22(1)
2.00 * * * 181(4) 127(3) 88(2) 77(2) 68(2) 46(1) 30(1) 22(1)
2.25 * * * 208(5) 127(3) 88(2) 77(2) 68(2) 61(2) 30(1) 22(1)
2.50 * * * * 150(4) 109(3) 77(2) 68(2) 61(2) 30(1) 22(1)
2.75 * * * * 173(5) 109(3) 95(3) 68(2) 61(2) 30(1) 22(1)
3.00 * * * * 195(6) 129(4) 95(3) 84(3) 61(2) 30(1) 22(1)
3.25 * * * * * 148(5) 112(4) 84(3) 61(2) 30(1) 22(1)
3.50 * * * * * 167(6) 112(4) 84(3) 76(3) 40(2) 22(1)
3.75 * * * * * 185(7) 129(5) 100(4) 76(3) 40(2) 22(1)
4.00 * * * * * * 146(6) 100(4) 89(4) 40(2) 22(1)
5.00 * * * * * * * 158(8) 116(6) 40(2) 30(2)
6.00 * * * * * * * * 179(11) 50(3) 30(2)
7.00 * * * * * * * * * 68(5) 37(3)

* Sample size is too large to be cost-effective for most audit applications.
Note: This table assumes a large population

Evaluation table for statistical sampling at 95 % confidence level: Upper limits of error as percentages

Annexure F

Sample size Actual number of Deviations found
0 1 2 3 4 5 6 7 8 9 10
25 11.3 17.6 * * * * * * * * *
30 9.5 14.9 19.6 * * * * * * * *
35 8.3 12.9 17.0 * * * * * * * *
40 7.3 11.4 15.0 18.3 * * * * * * *
45 6.5 10.2 13.4 16.4 19.2 * * * * * *
50 5.9 9.2 12.1 14.8 17.4 19.9 * * * * *
55 5.4 8.4 11.1 13.5 15.9 18.2 * * * * *
60 4.9 7.7 10.2 12.5 14.7 16.8 18.8 * * * *
65 4.6 7.1 9.4 11.5 13.6 15.5 17.4 19.3 * * *
70 4.2 6.6 8.8 10.8 12.6 14.5 16.3 18.0 19.7 * *
75 4.0 6.2 8.2 10.1 11.8 13.6 15.2 16.9 18.5 20.0 *
80 3.7 5.8 7.7 9.5 11.1 12.7 14.3 15.9 17.4 18.9 *
90 3.3 5.2 6.9 8.4 9.9 11.4 12.8 14.2 15.5 16.8 18.2
100 3.0 4.7 6.2 7.6 9.0 10.3 11.5 12.8 14.0 15.2 16.4
125 2.4 3.8 5.0 6.1 7.2 8.3 9.3 10.3 11.3 12.3 13.2
150 2.0 3.2 4.2 5.1 6.0 6.9 7.8 8.6 9.5 10.3 11.1
200 1.5 2.4 3.2 3.9 4.6 5.2 5.9 6.5 7.2 7.8 8.4

* Over 20 per cent
Note: This table presents upper limits as percentage. This table assumes a large population.

Presently Director (Performance Audit) – Office of the CAG of India, New Delhi

Go to the top