This reflection explores the gap between explaining and understanding artificial intelligence’s (AI) outcomes, focusing on the example of AI-driven medical diagnostics. Explainable AI (XAI) is fundamentally rooted in machine learning and probabilistic methods, which is why its relationship to domain-specific contexts, such as medical diagnostics or policymaking, as well as to social concepts, such as trust or accountability, remains ambiguous. This creates a gap between what is technically considered as explaining and what, in turn, constitutes the understanding of a model’s predictions and its concrete, situated implications. In our contribution, we draw on the philosophy of science and its contextual notion of scientific understanding to highlight the need for explainable algorithmic outcomes that enable actors to both explain and intervene in creating, applying or interpreting knowledge. Our reflection highlights the potential tension between XAI as providing insights into an AI model’s technical operations and explainability as part of a situated understanding of how the model works within specific social contexts. We believe that bringing the concept of scientific understanding to empirical AI ethics can help us rethink explainability and contribute to more responsible ways of making sense of and interacting with black-box models, particularly in medical diagnostics.

Generalized additive models (GAMs) are a leading model class for interpretable machine learning. GAMs were originally defined with smooth shape functions of the predictor variables and trained using smoothing splines. Recently, tree-based GAMs where shape functions are gradient-boosted ensembles of bagged trees were proposed, leaving the door open for the estimation of a broader class of shape functions (e.g. Explainable Boosting Machine (EBM)). In this paper, we introduce a competing three-step GAM learning approach where we combine (i) the knowledge of the way to split the covariates space brought by an additive tree model (ATM), (ii) an ensemble of predictive linear scores derived from generalized linear models (GLMs) using a binning strategy based on the ATM, and (iii) a final GLM to have a prediction model that ensures auto-calibration. Numerical experiments illustrate the competitive performances of our approach on several datasets compared to GAM with splines, EBM, or GLM with binarsity penalization. A case study in trade credit insurance is also provided.

The tail index is an important parameter that measures how extreme events occur. In many practical cases, this tail index depends on covariates. In this paper,we assume that it takes a finite number of values over a partition of the covariate space. This article proposes a tail index partition-based rules extraction method that is able to construct estimates of the partition subsets and estimates of the tail index values. The method combines two steps: first an additive tree ensemble based on the Gamma deviance is fitted, and second a hierarchical clustering with spatial constraints is used to estimate the subsets of the partition. We also propose a global tree surrogate model to approximate the partition-based rules while providing an explainable model from the initial covariates. Our procedure is illustrated on simulated data. A real case study on wind property damages caused by tornadoes is finally presented.