5 Risks of Automation & AI that Modern Auditors “Cannot Overlook”
Automation and Artificial Intelligence (AI) technologies have become key drivers of business in the current. Auditors (external and internal auditors), these advancements come with new forms of risks that are highly complex.
Here are 5 Risk of automation and AI that modern auditors “cannot overlook” to ensure their audits keep abreast of technological advancements.
1. Data quality and bias risks
The effectiveness and accuracy of AI and Automation systems depend on the data used to feed and train them.
- Risk: If the data used is of poor quality, inaccurate, or biased, the results processed by the AI will also be inaccurate and biased (garbage in, garbage out). For example, a personnel selection system might unintentionally exclude certain population groups if the training data has been biased in the past, or a loan approval system might be flawed and unfair.
- What auditors need watch out for: Verify the data selection process (data governance), data sources, and model testing for transparency and accuracy.
2. The “Black Box” problem and audit challenges
Advanced AI systems like Deep Learning often process information using logic too complex for humans to easily understand.
- Risk: When errors occur in the system, auditors, or even the developers themselves, may not be able to explain how the AI processed or made the chosen decision, resulting in a lack of a clear audit trail.
- What auditors need watch out for: Assess whether the organization has an Explainable AI (XAI) concept, or AI capable of explaining its thought processes, and whether it has adequate logging systems.
3. Cybersecurity and data breach risks
The use of automation and AI systems requires access to big data, including personal data (PDPA) or trade secrets.
- Risk: Bots (RPA) or AI may be compromised through data theft or hacking, allowing malicious actors to alter their source code or models, causing system malfunctions. Furthermore, feeding confidential organizational data into public AI (such as the free version of ChatGPT) can lead to a loss of data control and risk violating internal data policies or trade secret requirements.
- What auditors need watch out for: Reviewing system access controls, data leakage prevention measures, and the organization’s AI governance policy.
4. Risks of over-reliance on technology
When automation systems operate stably and quickly for extended periods, employees often become overly trusting and neglect oversight.
- Risk: This leads to “Automation Bias,” where complete reliance on the system is neglected by human oversight (Human-in-the-loop). If the system malfunctions due to changes in the business environment (Concept Drift), the damage can spread rapidly before anyone notices.
- What auditors need watch out for : Check for manual override processes or steps requiring human review and approval in high-risk areas.
5. Legal, regulatory compliance, and ethical risks
Currently, laws and regulations governing the use of AI are becoming increasingly stringent worldwide.
- Risk: Using AI that infringes on copyright (e.g., training AI with copyrighted images or text) or using AI in a way that violates ethical principles and data privacy laws can lead to lawsuits, massive fines, and severe reputational damage to the organization.
- What auditors need watch out for: Verify that the development and use of AI within the organization complies with relevant data privacy laws and regulations, both domestic and international, and that an AI Ethics Framework has been established within the organization.
The new role of auditors in the AI era: Moving beyond traditional document review and retrospective analysis, modern auditors must adapt to become “Proactive Risk Advisors”. This means being involved from the initial stages of designing and implementing automation and AI systems to ensure that controls are integrated into the technology from the outset (Control by Design).
In short, for modern auditors : our role isn’t to reject technology, but to build “trust but verify” to ensure the organization’s rapid growth will be based on security and transparency.
Sources:
- AI Risk Management and Ethics: NIST AI Risk Management Framework (AI RMF), National Institute of Standards and Technology (NIST), USA.
- Information Systems Audit and Transparency (XAI): Auditing AI Framework & COBIT, Information Systems Audit and Control Association (ISACA).
- The Proactive Role of the Auditor: Global Internal Audit Standards, The Institute of Internal Auditors (The IIA).
- Information Security Control and International Law: ISO/IEC 27001 (Information Security Management) and EU Artificial Intelligence Act (EU AI Act).
Author: Tanyalak Limothai
