In response to requests from auditors for clarity on applying the requirements of AS 1105, Audit Evidence, the PCAOB issued guidance that addresses the relevance and reliability of information from external sources that the auditor plans to use as audit evidence. The guidance also discusses the relationship between the quality and quantity of audit evidence in the guidance document.
AS 1105 outlines what constitutes audit evidence and establishes requirements regarding designing and performing audit procedures to obtain sufficient appropriate audit evidence. It has come to the PCAOB’s attention through its outreach efforts that the expanded use of information from sources external to the company is affecting the volume and nature of information available to auditors to use in the performance of audits.
In particular, advances in technology have improved accessibility and expanded the volume of information available to companies and their auditors from external sources. Some comes from traditional external sources of information such as regulatory agencies and industry data providers that now have interactive applications that can provide real-time industry data to companies. Newer, nontraditional external sources, such as web data aggregators and social media platforms, are more prevalent, the PCAOB noted, and provide information such as product reviews, weather patterns, and customer web traffic data to inform business and financial reporting decisions.
If the auditor plans to use this information as audit evidence, it must evaluate the relevance and reliability of the information, regardless of whether it has been used by the company in preparing the financial statements, the PCAOB stated.
Sufficiency and appropriateness. In the guidance, the PCAOB notes that an auditor is required to obtain sufficient appropriate audit evidence to provide a reasonable basis for the auditor’s opinion, where sufficiency is the measure of the quantity of the evidence, and appropriateness is the measure of its quality. The guidance states that in order to be appropriate, audit evidence must be both relevant and reliable in providing support for the auditor’s conclusions.
The concepts of relevance and reliability are interrelated, the guidance notes, and information that is more relevant but obtained from a less reliable source, or information that is less relevant but obtained from a more reliable source, may need to be supplemented to provide more persuasive audit evidence. The amount of evidence needed also depends on the purpose of the audit procedure, with substantive procedures and tests of controls requiring more persuasive evidence than risk assessment procedures, according to the guidance.
In general, as the risk of material misstatement increases, the amount of evidence that the auditor should obtain increases, as does the need for relevant and reliable audit evidence, the PCAOB notes. To supplement evidence that is less relevant or obtained from a less reliable source, auditors are increasingly turning to external sources and must evaluate both the relevance and reliability of that information, the PCAOB said.
Guidance on relevance. The guidance acknowledges that in some situations whether external information is relevant to the objective of the audit procedure performed may be readily apparent, but less so in others. The audit effort needed to evaluate the relevance of external information will vary.
The guidance provides the example of banks using historical loss information from other financial institutions to estimate current expected credit losses. According to the guidance, an auditor’s evaluation of the relevance of this external information could be informed by: (1) whether the loan products of the bank and other financial institutions are similar; (2) whether the bank’s loans and the other institutions’ loans were originated with similar underwriting standards; (3) whether the other financial institutions are similar to the bank in customer base; and (4) whether the borrowers have a similar geographic location.
To determine the nature and strength of any relationship between this information and the company’s transactions, and to substantiate conclusions reached, the auditor may need to perform additional procedures, according to the guidance. For example:
- A company could use customer reviews of its products from a social media website to monitor customer satisfaction and identify any emerging quality issues. This information could inform the auditor’s understanding of how the company collects information about potential quality problems and identifies a need for changes to warranty reserves. However, to determine whether social media reviews provide relevant evidence to support conclusions from substantive analytical procedures performed for a warranty reserve, the auditor would need to further understand how closely the negative customer reviews are correlated with product returns or warranty claims, taking into consideration the company’s business, the industry, and the nature of the company’s products.
- Some research suggests that weather data may be used to predict retail customer behavior and sales trends. However, before using the weather data in developing certain expectations, the auditor would need to understand the relationship between weather data and company activities to determine the relevance of the data to the audit objective. This may involve, among other things, comparing historical weather trends and historical trends in the company’s revenue.
The guidance indicates that the age of the external information and the time period it covers are also important in considering the information’s relevance. For example, in performing substantive analytical procedures over a utility company’s revenue, the relevance of census data to the auditor’s expectations of revenue and to achieving the desired objective of the procedure could vary depending on whether there have been significant expansions or contractions in the related population since the data was collected.
The guidance also warns that a certain type of information used as audit evidence in a prior audit may become less relevant in subsequent audits due to changes in the information or the account to which the evidence relates.
Guidance on reliability. According to the guidance, the reliability of audit evidence depends on the source and nature of the evidence and the circumstances under which it is obtained. The guidance provides multiple examples of factors that may affect the auditor’s evaluation of the reliability of external information, including these related to the source of the information: (1) expertise or reputation of the external source; (2) extent of regulatory oversight of the external source; and (3) relationship of the external source to the company.
Factors related to the nature of the information include whether the information has been originated, aggregated, or adjusted by the external source. The guidance notes that processing errors during the aggregation may reduce the reliability of the output, and that adjusted information may be more susceptible to processing error and bias than the original data.
Another relevant factor for the auditor is whether the information has been subject to review or verification, the guidance states. Information that has been subject to review or verification procedures would likely be more reliable than information that has not been reviewed or verified, according to the guidance.
Finally, the guidance advises that auditors should be aware of the circumstances under which information is obtained. Specifically, the auditor should consider whether the information was obtained directly by the auditor, because information obtained directly generally is more reliable than information obtained indirectly. Auditors also should weigh whether the information was obtained through a complex process, where the more complex the process, the greater the likelihood that a processing error may occur, the guidance notes.
Factors affecting reliability should be considered in combination, the guidance continued. In addition, if external information is subject to restrictions, limitations, or disclaimers, the auditor should evaluate the effect of the restrictions, limitations, or disclaimers on the reliability of that evidence. The guidance also states that if audit evidence obtained from more than one source is inconsistent with that obtained from another, or if the auditor has doubts about the reliability of external information, the auditor should perform the audit procedures necessary to resolve the matter and should determine the effect, if any, on other aspects of the audit.