EU Needs to Clearly Define the Differences Between Explanation and Understanding in AI Regulation of the Act
The European Union's proposed Artificial Intelligence Act (AIA) is set to revolutionise the AI landscape by emphasising transparency, explainability, and traceability, particularly for high-risk applications.
The AIA, while not explicitly defining interpretability, generally refers to the human ability to comprehend the logic behind AI decisions. Explainability, on the other hand, is a critical aspect of the Act, focusing on making AI decisions transparent and understandable. This is crucial in high-stakes sectors such as healthcare and finance.
High-risk AI systems, as per the Act, must be designed and developed in such a way that their operation is transparent enough for users to interpret the system's output. This transparency requirement extends to comprehensive disclosure about AI system capabilities, limitations, and potential risks, including labeling AI-generated content.
The Act also mandates the use of techniques that provide clear explanations for AI decisions, such as feature attribution methods, model interpretability techniques, or model-agnostic explanations. Data traceability is another key requirement, ensuring that AI outputs can be traced back to their source data, enhancing accountability and transparency.
However, the AIA demonstrates a potential inconsistency between its call for "explainable" AI and its requirement for users to "interpret" AI systems. To address this, the EU should specify that providers of high-risk AI systems should clarify how a system can be explained as part of the transparency obligations in Article 16.
It's important to note that requiring all high-risk systems to be interpretable may not be the correct approach, as high performance is preferable in some instances. For example, in critical infrastructure like traffic systems and public services like emergency first response systems, high performance is more important than interpretability.
The distinction between explainable and interpretable AI systems is significant, as their requirements differ substantially. While explainable systems can be understood by humans, interpretable systems allow designers to understand how features and weights determine the output.
The EU should outline the technical measures by which a provider can demonstrate a system's interpretability. However, it should refrain from obliging any particular explanation method for high-risk AI systems, as algorithms used to formulate explanations are not yet perfect and may be unsuited to adversarial contexts.
In conclusion, the EU's AIA aims to create a trustworthy AI ecosystem by mandating transparency, explainability, and traceability, particularly for high-risk AI applications. By doing so, it hopes to prevent potential misuse and ensure accountability, while encouraging innovation in the AI sector.
The AIA mentions the human ability to comprehend AI decisions as interpretability, but the Act's focus is on making decisions transparent and understandable, known as explainability. High-risk AI systems must be designed for transparent output interpretation and disclose their capabilities, limitations, and risks extensively. The AIA also demands clear explanations for AI decisions through methods like feature attribution, model interpretability, or model-agnostic explanations. However, the Act's requirement for users to interpret AI systems potentially contradicts its call for "explainable" AI, necessitating clarification from providers regarding demonstrating a system's interpretability. The EU should avoid obliging a specific explanation method for high-risk AI systems, as algorithms used may not be suitable in adversarial contexts.
