aiaudit.org

We are a group of humans who audit AI systems.

Modern AI systems based on deep learning, reinforcement learning or hybrids thereof constitute a flexible, complex and often opaque technology. Limits in our understanding of an AI system’s behavior constitute risks for system failure. Hence, the identification of failure modes in AI systems is an important pre-requisite for their reliable deployment to real-world settings.

aiaudit.org is a cooperative of humans working at the intersection of machine learning research, regulation, software development and application domains. We design methods, processes and standardization contributing towards AI technology that can be trusted for use in real-world applications. For that purpose, many of us contribute their expertise to standardization efforts such as the ITU/WHO Focus Group on Artificial Intelligence for Health or Xavier AI Team of the FDA’s Digital Health Center of Excellence.

This site is the group’s interface to present and organize its work. We maintain information on current projects and their coordinates on the project sites. We invite you to take a look. If you are interested to contribute or learn more about our modus operandi please consult the join page. We are committed to equitable collaboration. Project groups form in a self-organized way and are open to interested persons from research, companies, the general public or otherwise.

Calendar

Selected outputs

  1. ITU/WHO
    FG-AI4H Open Code Initiative - Evaluation and Reporting Package
    Schörverth, Elora, Vogler, Steffen, Balachandran, Pradeep, Leite, Alixandro Werneck, Li, Danny Xie, Ali, Kamran, Garcia, , Schneider, Dominik, Krois, Joachim, Lecoultre, Marc, Iyer, Shobha, Choudhary, Shruti, and Oala, Luis
    In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting K 2021
  2. ITU/WHO
    Good practices for health applications of machine learning: Considerations for manufacturers and regulators
    Johner, Christian, Balachandran, Pradeep, Oala, Luis, Lee, Aaron .Y., Leite, Alixandro Werneck, Murchison, Andrew, Lin, Anle, Molnar, Christoph, Rumball-Smith, Juliet, Baird, Pat, Goldschmidt, Peter. G., Quartarolo, Pierre, Xu, Shan, Piechottka, Sven, and Hornberger, Zack
    In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting K 2021
  3. ITU/WHO
    Data and artificial intelligence assessment methods (DAISAM) Audit Reporting Template
    Verks, Boris, and Oala, Luis
    In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting J 2020
  4. ITU/WHO
    Data and artificial intelligence assessment methods (DAISAM) reference
    Oala, Luis, Balachandran, Pradeep, Cabitza, Federico, Calderon Ramirez, Saul, Chiavegatto Filho, Alexandre, Eitel, Fabian, Extermann, Jérôme, Fehr, Jana, Ghozzi, Stephane, Gilli, Luca, Jaramillo-Gutierrez, Giovanna, Kester, Quist-Aphetsi, Kurapati, Shalini, Konigorski, Stefan, Krois, Joachim, Lippert, Christoph, Martin, Jörg, Merola, Alberto, Murchison, Andrew, Niehaus, Sebastian, Ritter, Kerstin, Samek, Wojciech, Sanguinetti, Bruno, Schwerk, Anne, and Srinivasan, Vignesh
    In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting I 2020