Knight, Determining Audit Findings

DETERMINING AUDIT FINDINGS Frank Knight [email protected] Consultant, Canada This article explores some of the challenges in developing audit findings. The message is that there are more ways than one to approach the audit and that the auditor should not be constrained by a rigid application of the standard methodology. We present here two alternative approaches. Simply stated they are: Traditional Model: What should be; what is; finding results from identification of a gap or difference. Alternative (or Converse) Model: What is; what could be; finding results from judgement of whether appropriate / not appropriate in the circumstances.

Traditional Model The standard approach is to compare what is in place against a standard or criterion. The auditor defines what should be in place to manage a function. In financial audit, the audit refers to GAAP (generally accepted accounting practices). In performance audit, the auditors develop audit criteria against which the management systems, procedures and practices can be compared. The auditor can then determine whether the organization is being managed according to these standards. In this way, the auditor generates findings where there is a gap between what should be in place and what is in fact in place. The auditor thus identifies a finding as a situation where there is a discrepancy between what should be (the criterion) and what is. In this way findings are generated. This is the theory normally presented in audit methodology. Furthermore, it is commonly believed that without a criterion, a finding cannot exist. In practice, this is not a reasonable assumption. Especially in performance audit, it is possible to encounter situations that had not been anticipated. Or, the experience of the auditor is insufficient to develop a sufficiently complete set of criteria to apply to the area being audited. During the audit process, the auditor may sometimes encounter, or observe, a situation that is obviously, or apparently, wrong but for which, no prior criterion had been developed. Some auditors then go back and “invent” the appropriate criterion. This is not intellectually honest. International Journal on Governmental Financial Management – 2008

133

Where an unsatisfactory situation is encountered where no prior criterion has been developed, the auditor should recognize, nevertheless that a reportable observation has been made. When discussing this type of observation with the management of the operations, it should be explained (if the original set of criteria had been given to management prior to the start of the audit) that no prior criterion had been developed but the concern is still a reality. Management should be persuaded that there is a concern, or, if there is a good explanation for the problem, the observation should take this into account.

Alternative (or Converse) Model Another approach, as often applied in social studies, is to examine what is in place, i.e. how the managers are managing their business. With this approach, the auditor notes situations where results are not being achieved, where errors are occurring, where there is waste or mismanagement. Also, of course, the auditor notes, and in some cases provides assurances, where the operations are carried out successfully. With this approach the auditor still needs to know what is usual practice and should always develop appropriate criteria. This knowledge and set of “expectations” is needed to conduct the audit. The emphasis of the audit work, however, is on understanding how the managers have chosen (or been forced) to manage the particular situation. The concept here is that if it works (and that there are no serious risks that it may fail in a particular situation) then the auditor may accept that it is appropriate even if it is different from normal practice. At the same time, the auditor should understand why it may differ from the criteria developed before-hand or, where no criterion had been developed to address the situation, why an additional criterion is required. There can be many reasons why it does not conform to normal practice. In discussion with management, the auditor confirms that there are problems or reasons for doing things differently, and determines the cause(s) of this condition. Thus the auditor confirms the situation and determines what observations are appropriate in the circumstances. Then the auditor concludes what changes should be made to improve the management structure, processes and practices and makes recommendations. Often in governments that are short of funds the “best” management practices may not be realistic. For the auditor to recommend more funds (or suggest systems that would be beyond the budget of the organization) will not be helpful in the situation. The auditor should suggest perhaps different methods, or different allocation of effort to improve overall performance. 134

International Journal on Governmental Financial Management – 2008

The Difference In practice, both directions of logic are used. Yet, it is important that the auditor is aware of the alternative approaches. Understanding the reality and adjusting expectations accordingly is more realistic, and certainly more useful, than limiting the audit process to one of comparing “what is“ with “what should be” to produce the audit findings. Not Necessary to Have Every Criterion Addressed and Every Finding Reported The “traditional” approach is often combined with the insistence that every criterion should be addressed and a finding reported, either that the practice is consistent with the criterion (a positive finding) or that there is a deficiency (a negative finding). The difficulty with this is two fold: The auditor is forced into reporting matters that may not be of significance; and/or In some cases, extensive evidence (e.g. a large sample size) is required to ensure the validity of a positive finding (i.e. the provision of assurance). There should be continual review of the audit work to ensure a trade-off between allocating scarce audit resources to insignificant areas and obtaining sufficient evidence to arrive at supportable conclusions for significant findings. This means that the auditor should not always attempt to produce a complete set of findings. It is more important to report on a few key audit concerns than provide a “complete” audit coverage. Furthermore, many small findings can detract from the main message the auditor wishes to provide management. Some managers want to be provided all the detailed findings, while others want a concise report that conveys just the important message. This is a reporting issue but from the findings perspective, the message is that many detailed findings are not necessary. In most cases, where the report is more than say 15 pages long, there should be an Executive Summary. This should provide a clear and meaningful message that has limited details. It should provide a summary of the audit objectives and scope, the main findings, the significant conclusion(s) and a list of the more significant recommendations contained in the main body of the report. Even in the main body of the report, it is preferable to keep the communications simple and clear, placing detailed calculations and listings of detailed observations in appendices."

International Journal on Governmental Financial Management – 2008

135

Clusters of Findings The ability to cluster minor and/or detailed findings into a more generalized finding is a valuable skill the auditor should develop. For example, the auditor may discover: errors in reports / late in issuing reports / poor planning process for issuing reports / no knowledge of what information managers need / lack of clear/consistent definitions used in reports. These in themselves may not be very significant findings but collectively they indicate serious problems in the management of the reporting function. The higher level finding can be expressed as “inadequate reporting of the state of operations”. The conclusion can then be stated as “management are unable to rely on the reports provided them”. At all times, the auditor must ensure that the lower level findings are sufficient to be able to support the higher level finding.

Scope of the Audit Coverage The INTOSAI standards state that only evidence related to the originally identified audit problem should be collected28. This position raises some interesting ethical issues. Presumably the intention is to not “conduct a witch hunt”. First, the auditor should never conduct the work in a subjective or personal manner. There should be no “hidden agenda” or a wish to blame an individual manager. Nevertheless limiting the collection of information solely with regard to the original audit scope presents some problems. As with a financial (or attest) audit, there are very clear procedures to follow and well defined scope of audit. Nevertheless, in for example a financial audit, should any evidence of fraud, or suspected fraud, come to light, it is the duty of the auditor to extend coverage as appropriate in these circumstances. Similarly, it is suggested that if any unacceptable situation is encountered during a performance audit, the evidence should not be ignored. Proper procedures should be followed of course. Where evidence is found outside of the original scope or subject of audit, the auditor should consult with his/her audit manager and then with senior Audit Office management. After that, if it is still considered appropriate to pursue the area further, the contact at the audit entity should be consulted and the situation explained. At this point, it may be agreed that the area should be included in the audit.

Need for Judgement In practice, the process of determining findings can vary considerably. Some findings are a clear “right” or “wrong”. Often, however, the findings are not that simple. 28

According to Code of Ethics and Auditing Standards, INTOSAI, 2001, page 70, Findings are "the specific evidence gathered by the auditor to satisfy the audit objectives". This means that only findings that are directly or indirectly related to the selected audit problem(s) are relevant in the audit. 136

International Journal on Governmental Financial Management – 2008

Some activities in audit are fairly mechanical. A set of calculations are made to determine whether the reported data are accurate. This could be in support of an assessment of the value of assets, the statement of revenue, the level of efficiency, or the amount of time a vehicle spent in maintenance and repair. Other activities can be highly judgemental. Example: Analysis of Investment Options In an investment analysis, the auditor needs to determine whether an appropriate set of options was considered before making the decision of which option to select. In this situation, the auditor would not accept: do nothing; spend at least ten times as much; and accept option presented. This would not constitute a realistic set of options for analysis. On the other hand, if the auditor wishes to observe that insufficient options were considered, he/she must be prepared to put forward realistic and meaningful options that should/could have been assessed in the particular situation. Example: Responsibilities Cleary Allocated It is common to include a criterion that states: “roles and responsibilities should be clearly allocated”. It is generally a sound management practice to allocate responsibilities clearly and hold one individual responsible for certain activities. Another style of management, however, contradicts this principle. The “duplicate tasking” manager tasks more than one individual to carry out the same work. Then the manager receives more than one result and can select the preferred result (or even have a competition between the staff). On the basis of the pre-defined criterion, this “duplicate tasking” manager is not carrying out the job properly. The finding is that responsibilities have not been assigned clearly. Alternatively, the auditor documents the process followed by the “duplicate tasking” manager and before coming to a clear finding, explores the consequence of this manager’s approach. If morale suffers, and there is serious waste of effort then the “duplicate tasking” approach could be considered inappropriate. Alternatively, the auditor may find that the “duplicate tasking” manager receives much better information and makes more informed decisions with the duplication of effort. Example: Strategic Planning Process in Place The auditor must guard against acceptance of a process in place rather than testing whether the process is meaningful. An interesting example was an audit of military planning. The first auditor examined the systems and processes in place for strategic planning and concluded that these were well presented and applied. Another auditor came along and asked a simple question: “How would the department integrate the reserves with the regulars in the case of war?” This simple, but significant, question had not been addressed and the department agreed it was a key consideration. Thus the strategic process in place International Journal on Governmental Financial Management – 2008

137

may have been impressive as a process but had failed to recognize significant scenarios that the strategic process had to address. Audit has to rely on sound professional knowledge and application of appropriate methodology but the successful auditor cannot operate without good judgement and an open and enquiring mind.

138

International Journal on Governmental Financial Management – 2008