aiaudit.org

Modern AI systems based on deep learning, reinforcement learning or hybrids thereof constitute a flexible, complex and often opaque technology. Limits in our understanding of an AI system’s behavior constitute risks for system failure. Hence, the identification of failure modes in AI systems is an important pre-requisite for their reliable deployment to real-world settings.

aiaudit.org is a cooperative of humans working at the intersection of machine learning research, regulation, software development and application domains. We design methods, processes and standardization contributing towards AI technology that can be trusted for use in real-world applications.

This site is the group’s interface to present and organize its work. We maintain information on current projects below. We are committed to equitable collaboration. Project groups form in a self-organized way and are open to interested persons from research, companies, the general public or otherwise.

Core activities

ITU/WHO AI4H Audit Platform

We develop and maintain an audit platform for health AIs on UN ITU AWS infrastructure.

Journal for ML4H Auditing

In partnership with Springer Nature, we maintain a peer-reviewed venue for deliberation on the methods, processes and tools needed to manage the particular risks of ML4H systems.

ML4H Trial Audits

With a team of more than 40 experts spanning machine learning, clinical, regulatory and ethical expertise, we are running trial audits for 11 health AIs from the ITU/WHO FG-AI4H.