iSixSigma

Sampling Plans for Auditing Transactions

Six Sigma – iSixSigma Forums Operations Finance Sampling Plans for Auditing Transactions

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #54389

    gaurav k
    Participant

    Hi!
    Say I have a 1000 transactions (for eg. Employee travel reimbursements) that I wish to audit. How can I decide on a sampling plan for auditing the same? I’m looking for a rational and defensible basis for deciding sampling sizes.
    I’m familiar with acceptance sampling and have also read AICPA guidelines for audit sampling, but I’m more confused now than before! My doubts are in the following areas:
    1. AICPA guidelines calculate sample sizes by assuming a large population, and so suggests an absolute number of transactions to audit. What if I want a % figure, since my sampling may be on a periodic basis for every 1000 transactions?
    2. Each transaction consists of a series of steps, with varying business risks associated with failure of each step. How can I design a plan that takes into account the sub-steps as well?
    3. Does anyone have an excel template that helps with the calculations needed to answer the above questions?
    I’d greatly appreciate some assistance in sorting this out. Sorry for being so verbose but I thought it best to put all I had on my mind out here!

    Regards,
    Gaurav

    0
    #195175

    gaurav k
    Participant

    Was this the right forum to post the above question? If not, can anyone suggest where I could post the question?
    Thanks!

    0
    #195177

    Prabhu V
    Participant

    Hi Gaurav,

    To answer your question in general, you can apply the sample size calculation using the discrete data method.

    However, you should have some pre-work data for this method (say for example, from the previous analysis about 10% of transactions were erroneous)

    If you have similar situation, then you can apply the discrete data logics as follows:

    N=(Z/E)^2*p(1-p)

    Where

    N – Sample size
    Z – Constant for confidence level (like 1.645 for 90% confidence, 1.96 for 95% confidence, 2.575 – 99% confidence)
    E – Precision or Error (level of precision desired from the sample in units of proportion)
    p- Proportion

    in your case, like you wish to have the erroneous transactions to 5% from 10% (previous analysis data) then, you calculation will be
    N = (1.96/0.05)^2*.10(1-0.10)
    N ~ 139 transactions considering that 95% confidence level being expected from the audit.
    I hope this explanation can help you.

    Pls feel free to post your queries

    All the best!!

    Regards

    Prabhu V.

    0
    #195195

    gaurav k
    Participant

    Hi Prabhu,
    Thank you for your inputs! Please see if I’ve got this right. The above calculation for sample size tells me: “How many samples should I pick so that I’m 95% confident that the % of defectives in my population is not different from the % of defectives in my sample by more than 5 percent points.” Is that right?

    I’m not clear about this though:
    Based on my audit results, if i find less of more % of defective transactions in my sample, then do i have to re-calculate my population proportion of defectives?

    2. Do acceptance quality plans follow the same principle?

    0
    #195203

    Prabhu V
    Participant

    Gaurav,

    Can you elaborate your queries for better understanding (Hoping that I have given all the assumptions used in the above example in detail.)

    Regards

    0
    #195204

    Chris Seider
    Participant

    Are you sampling to get data or as a control measure?

    0
    #195211

    gaurav k
    Participant

    I’ve been reading up on the issue and also trying to simplify the problem in my head. Let me try to restate what I need (I think!).
    The purpose of the audit is to ensure that processes are being performed as defined; it’s a control measure if this is what you’ve meant, Chris. The purpose is not to estimate any population parameter by taking samples.
    Prabhu, my main doubt with calculating a sample size based on what your formulae, is that I am trying to pick a sample where the %non-conformities in the sample reflect the %non-conformities in the population. When it comes to auditing for controls, could you explain how this approach would serve the purpose?
    Thannks for helping me think through this!

    0
    #195216

    Prabhu V
    Participant

    Gaurav,
    After studying the entire thread, I would like to provide the following:-
    a) The purpose of audit which you would like to introduce is to ensure proper guidelines being followed in each transaction (i.e the system should not be misused)
    b) Since the audit is going to be iterative once introduced it will create a awareness among the employees that there is a chance of misusing the system can be monitored and altered (like creating a ticket collector/checker mechanism in public transport like trains and buses)
    As an initial step, if you’re agreeing the above points means you can do as below:-
    i) First you can collect the base line data for erroneous transactions from the concerned based on their experience and history (say about 10% or 15%)
    ii) Once the baseline data confirmed means you can perform the sample size calculation as mentioned above for first 1000 transactions and find the exact erroneous transaction for 99% confidence level (if critical).
    iii) The outcome of step ii) is say 10% erroneous transactions then you can iterate the audit process for another 1000 transactions to get the trend after intimating the step ii) outcomes to the management/employees.
    iv) If the outcome of step iii) can be lesser than step ii) means, the audit yield good results on providing a control in the system.
    Of course, I have not estimated about the resources required and complexity involved in the audit process etc., into account on above (practical issues).
    Hoping your doubts is clarified.
    Regards
    Prabhu V.

    0
    #195230

    MBBinWI
    Participant

    @gaurav777 – You are really looking to accept/reject a lot (the daily, weekly or monthly volume of transactions) based on a sample of those transactions. Reject would indicate the controls are not working, accept that they are.

    If you have Minitab, you can do this via the Stat > Quality Tools > Acceptance Sampling by Attributes. You will need to determine your AQL and RQL (acceptable quality level and rejectable quality level). The definitions of these from Minitab help are below.

    Acceptable quality level (AQL): Enter a number to represent the largest number of defectives or defects in a process that will still be considered acceptable. Typically, a sampling plan is designed to give a high probability of acceptance at the AQL.
    You must specify the AQL when you create a sampling plan, but it is not required for comparing sampling plans. The AQL must be consistent with measurement units. For example, with percent defectives 0<AQL<100; with proportion defective 0<AQL<1; and with defectives per million 0<AQL<1,000,000.

    Rejectable quality level (RQL or LTPD): Enter a number to represent the number of defectives or defects in an individual lot that you are willing to tolerate. Typically, a sampling plan is designed to give a low probability of acceptance at the RQL.
    You must specify the RQL when you create a sampling plan, but it is not required for comparing sampling plans. The RQL must be entered consistently with measurement units and must be larger than the AQL. For example, with percent defectives AQL<RQL<100; with proportion defective AQL<RQL<1; and with defectives per million AQL<RQL<1,000,000.

    Lot size: Enter a number to represent the lot size or batch size of the entire shipment that you will accept or reject based on sampling results. You don't need to specify a lot size if you specify the AQL, RQL, and alpha and beta risks; however, Minitab requires the lot size to calculate the AOQ curve and the ATI curve.

    For Creating Sampling Plans
    Producer's risk (Alpha): Enter a value between 0 and 1 to represent alpha. 1-alpha represents the desired probability of accepting a lot at the AQL, which is necessary for creating the sampling plan.

    Consumer's risk (Beta): Enter a value between 0 and 1-alpha to represent beta. Beta represents the desired probability of accepting a lot at the RQL, which is necessary for creating the sampling plan.

    Often the AQL and RQL is set too low, driving the sampling rate very high. Understanding the costs of sampling and costs of incorrect data (in your case by process item/step) can help to choose these more appropriately.

    0
    #195270

    gaurav k
    Participant

    Dear All,

    Thank you so much for your inputs! I’ve been spending a lot of time reading up and hopefully I’ve managed to tie things together.
    Yes, I’ll be using attribute sampling principles, though not exactly to “Accept” or “Reject” transaction lots. My objective is to quantify chances of detecting errors in the sample. I would then look at cost constraints to determine how much I can maximise my chances.
    The main objective with my sampling plan is to determine “How many samples (n) should I pick out of a lot of N, with a historical error rate (can also be read as tolerable error rate) of t%, so that I’m 95% confident that I’ll be able to pick up at least X defectives in my sample.”
    I’m working on an excel template that would calculate the sample size. I’ve used the hypergeometric distribution function to calculate sample sizes. Mainly because it gives me exact probabilities, but also the fact that my error rates and population sizes are such that I cannot approximate the probability distributions to binomial or normal distributions.
    The template calculates values for n, such that: 1-CumulativeHypergeometric(X-1, n, tN, N) > 95%
    Prabhu’s formula is also one way of calculating sample sizes, but holds good only when n*t >10.
    @MBBinWI, I’ll try reading up on acceptance sampling and see if there’s anything else i need to consider other than what I’ve mentioned above

    0
    #196325

    Kimmy Burgess
    Guest

    In an auditing environment, first and foremost, you must remember that professional judgement is the most important factor before picking up an approach. As far as AICPA is concerned, it permits both statistical and non-statistical tools to be used. Depending on the risk of material misstatement, you may choose the tool which best addresses the risk. For a routine transaction like employee reimbursements, I would go for random sampling.

    0
Viewing 11 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.