EU AI Act: High-Risk AI Systems

Product manufacturers and certain sectors of the technology industry that use artificial intelligence in their products are the key targets of the EU AI Act. These categories of use of AI are specifically called out as high-risk and they will be subject to a significant new conformity assessment regime.

The AI Act broadly splits high-risk AI uses into two categories – one for physical products which are listed in Annex II to the Act and the other, more software based, is set out in Annex III.

Those in Annex II include machinery, toys and medical devices and those in Annex III include:

  1. Critical infrastructure like wind and water where the use of AI could put people’s life and health at risk;
  2. Use in grading in educational and vocational settings;
  3. Use in recruitment or performance in the employment space; and
  4. Access to essential private and public services like financial services via credit scoring apps.

Before these high-risk AI systems can be put on the EU market, they are subject to strict ‘conformity assessments’, which determine whether the system will meet all requirements in the Act. Providers dealing in the products listed in Annex II will conduct these assessments and report to regulated notified bodies, while providers of the technologies covered by Annex III will self asses to the same requirements.

Some examples of the requirements include:

  • Establishing and maintaining a risk management system;
  • Ensuring training, validation, and testing data sets are subject to appropriate data governance and management practices; and
  • Putting in place measures to guarantee human oversight of these high-risk AI systems.

Many providers of the types of products listed in Annex II will have some experience of dealing with a notified body and a conformity assessment regime. However, those in the technology sector falling into Annex III will probably not, and they will need to invest more in resources and expertise at the outset.

For more information on the EU AI Act or its impacts, contact a member of our dedicated Artificial Intelligence team.

The content of these articles are provided for information purposes only and does not constitute legal or other advice.