2021

1. ITU/WHO

   Good practices for health applications of machine learning: Considerations for manufacturers and regulators

   Johner, Christian, Balachandran, Pradeep, Oala, Luis, Lee, Aaron .Y., Leite, Alixandro Werneck, Murchison, Andrew, Lin, Anle, Molnar, Christoph, Rumball-Smith, Juliet, Baird, Pat, Goldschmidt, Peter. G., Quartarolo, Pierre, Xu, Shan, Piechottka, Sven, and Hornberger, Zack

   In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting K 2021

   Abs HTML PDF

   This document contains the latest draft of the FG-AI4H deliverable DEL02.2 "Good practices for health applications of machine learning: Considerations for manufacturers and regulators". This deliverable defines a set of guidelines intended to serve the AI solution developers/manufacturers on how to do conduct a comprehensive requirements analysis and to streamline the conformity assessment procedures to ensure regulatory compliance for the AI based Medical Devices (AI/ML-MD).

2. ITU/WHO

   FG-AI4H Open Code Initiative - Evaluation and Reporting Package

   Schörverth, Elora, Vogler, Steffen, Balachandran, Pradeep, Leite, Alixandro Werneck, Li, Danny Xie, Ali, Kamran, Garcia, , Schneider, Dominik, Krois, Joachim, Lecoultre, Marc, Iyer, Shobha, Choudhary, Shruti, and Oala, Luis

   In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting K 2021

   Abs HTML PDF Code

   Presentation of the software components developed by WG-DAISAM for automating the FG-AI4H evaluation process.

2020

1. ITU/WHO

   Data and artificial intelligence assessment methods (DAISAM) reference

   Oala, Luis, Balachandran, Pradeep, Cabitza, Federico, Calderon Ramirez, Saul, Chiavegatto Filho, Alexandre, Eitel, Fabian, Extermann, Jérôme, Fehr, Jana, Ghozzi, Stephane, Gilli, Luca, Jaramillo-Gutierrez, Giovanna, Kester, Quist-Aphetsi, Kurapati, Shalini, Konigorski, Stefan, Krois, Joachim, Lippert, Christoph, Martin, Jörg, Merola, Alberto, Murchison, Andrew, Niehaus, Sebastian, Ritter, Kerstin, Samek, Wojciech, Sanguinetti, Bruno, Schwerk, Anne, and Srinivasan, Vignesh

   In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting I 2020

   Abs HTML PDF

   This document, Data and artificial intelligence assessment methods (DAISAM) reference, is the reference collection of WG-DAISAM for assessment methods of data and artificial intelligence quality evaluation. This document also constitutes subsection 7.3 of the FG-AI4H deliverable 7.

2. ITU/WHO

   Data and artificial intelligence assessment methods (DAISAM) Audit Reporting Template

   Verks, Boris, and Oala, Luis

   In Proceedings of the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting J 2020

   Abs HTML PDF

   Standardized templates to report results for the assessment processes developed by WG-DAISAM. In this version the, template comprises three elements: Data Specification Sheet, ML Model Specification Sheet and ML Model Summary Findings.

2021

1. BVM Oral

   Interval Neural Networks as Instability Detectors for Image Reconstructions

   Macdonald, Jan, März, Maximilian, Oala, Luis, and Samek, Wojciech

   In Bildverarbeitung für die Medizin 2021 2021

   Abs HTML PDF

   This work investigates the detection of instabilities that may occur when utilizing deep learning models for image reconstruction tasks. Although neural networks often empirically outperform traditional reconstruction methods, their usage for sensitive medical applications remains controversial. Indeed, in a recent series of works, it has been demonstrated that deep learning approaches are susceptible to various types of instabilities, caused for instance by adversarial noise or out-ofdistribution features. It is argued that this phenomenon can be observed regardless of the underlying architecture and that there is no easy remedy. Based on this insight, the present work demonstrates, how uncertainty quantification methods can be employed as instability detectors. In particular, it is shown that the recently proposed Interval Neural Networks are highly effective in revealing instabilities of reconstructions. Such an ability is crucial to ensure a safe use of deep learning-based methods for medical image reconstruction.

2020

1. NeurIPS Spotlight
   Top 10%

   ML4H Auditing: From Paper to Practice

   Oala, Luis, Fehr, Jana, Gilli, Luca, Balachandran, Pradeep, Leite, Alixandro Werneck, Calderon-Ramirez, Saul, Li, Danny Xie, Nobis, Gabriel, Alvarado, Erick Alejandro Munoz, Jaramillo-Gutierrez, Giovanna, Matek, Christian, Shroff, Arun, Kherif, Ferath, Sanguinetti, Bruno, and Wiegand, Thomas

   In Proceedings of the Machine Learning for Health NeurIPS Workshop 2020

   Abs HTML PDF Slides Video

   Healthcare systems are currently adapting to digital technologies, producing large quantities of novel data. Based on these data, machine-learning algorithms have been developed to support practitioners in labor-intensive workflows such as diagnosis, prognosis, triage or treatment of disease. However, their translation into medical practice is often hampered by a lack of careful evaluation in different settings. Efforts have started worldwide to establish guidelines for evaluating machine learning for health (ML4H) tools, highlighting the necessity to evaluate models for bias, interpretability, robustness, and possible failure modes. However, testing and adopting these guidelines in practice remains an open challenge. In this work, we target the paper-to-practice gap by applying an ML4H audit framework proposed by the ITU/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) to three use cases: diagnostic prediction of diabetic retinopathy, diagnostic prediction of Alzheimer’s disease, and cytomorphologic classification for leukemia diagnostics. The assessment comprises dimensions such as bias, interpretability, and robustness. Our results highlight the importance of fine-grained and caseadapted quality assessment, provide support for incorporating proposed quality assessment considerations of ML4H during the entire development life cycle, and suggest improvements for future ML4H reference evaluation frameworks.

2. ICML Spotlight
   Top 10%

   Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty

   Oala, Luis, Heiß, Cosmas, Macdonald, Jan, März, Maximilian, Samek, Wojciech, and Kutyniok, Gitta

   In ICML 2020 Workshop on Uncertainty & Robustness in Deep Learning 2020

   Abs HTML PDF Slides Video

   The quantitative detection of failure modes is important for making deep neural networks reliable and usable at scale. We consider three examples for failure modes in image reconstruction problems and demonstrate the potential of uncertainty quantification as a fine-grained alarm system. We propose a deterministic, modular and lightweight approach, called Interval Neural Networks, that produces fast and easy to interpret uncertainty scores which improve the detection of failure modes across four out of five image reconstruction experiments.

2021

2020

1. arXiv

   Interval Neural Networks: Uncertainty Scores

   Oala, Luis, Heiß, Cosmas, Macdonald, Jan, März, Maximilian, Samek, Wojciech, and Kutyniok, Gitta

   2020

   Abs HTML PDF

   We propose a fast, non-Bayesian method for producing uncertainty scores in the output of pre-trained deep neural networks (DNNs) using a data-driven interval propagating network. This interval neural network (INN) has interval valued parameters and propagates its input using interval arithmetic. The INN produces sensible lower and upper bounds encompassing the ground truth. We provide theoretical justification for the validity of these bounds. Furthermore, its asymmetric uncertainty scores offer additional, directional information beyond what Gaussian-based, symmetric variance estimation can provide. We find that noise in the data is adequately captured by the intervals produced with our method. In numerical experiments on an image reconstruction task, we demonstrate the practical utility of INNs as a proxy for the prediction error in comparison to two state-of-the-art uncertainty quantification methods. In summary, INNs produce fast, theoretically justified uncertainty scores for DNNs that are easy to interpret, come with added information and pose as improved error proxies - features that may prove useful in advancing the usability of DNNs especially in sensitive applications such as health care.

2. arXiv

   MixMOOD: A systematic approach to class distribution mismatch in semi-supervised learning using deep dataset dissimilarity measures

   Calderon-Ramirez, Saul, Oala, Luis, Torrents-Barrena, Jordina, Yang, Shengxiang, Moemeni, Armaghan, Samek, Wojciech, and Molina-Cabello, Miguel A.

   2020

   Abs HTML PDF

   In this work, we propose MixMOOD - a systematic approach to mitigate effect of class distribution mismatch in semi-supervised deep learning (SSDL) with MixMatch. This work is divided into two components: (i) an extensive out of distribution (OOD) ablation test bed for SSDL and (ii) a quantitative unlabelled dataset selection heuristic referred to as MixMOOD. In the first part, we analyze the sensitivity of MixMatch accuracy under 90 different distribution mismatch scenarios across three multi-class classification tasks. These are designed to systematically understand how OOD unlabelled data affects MixMatch performance. In the second part, we propose an efficient and effective method, called deep dataset dissimilarity measures (DeDiMs), to compare labelled and unlabelled datasets. The proposed DeDiMs are quick to evaluate and model agnostic. They use the feature space of a generic Wide-ResNet and can be applied prior to learning. Our test results reveal that supposed semantic similarity between labelled and unlabelled data is not a good heuristic for unlabelled data selection. In contrast, strong correlation between MixMatch accuracy and the proposed DeDiMs allow us to quantitatively rank different unlabelled datasets ante hoc according to expected MixMatch accuracy. This is what we call MixMOOD. Furthermore, we argue that the MixMOOD approach can aid to standardize the evaluation of different semi-supervised learning techniques under real world scenarios involving out of distribution data.