To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
In this chapter, we consider nonparametric regression when we have more than one feature. First, we show how the methods in Chapter 6 can be extended to handle this case. Then, we consider additive regression, regression trees, and random forests. Another estimator based on neural nets is discussed in Chapter 12.
The COVID-19 pandemic exposed individuals to numerous psychosocial and health-related stressors associated with adjustment disorder (AjD) symptoms, yet it remains unclear which factors are most predictive.
Methods
Using mixed-effects regression random forests (MERF), a machine learning approach that combines random forests with mixed-effects regressions, we analyzed longitudinal data from 15,155 adults across 11 European countries collected at three time points between June 2020 and January 2022. We evaluated 245 candidate predictors, including sociodemographic, pandemic-related, and health-related factors, for their relative importance in predicting AjD symptoms (ADNM-8).
Results
The seven most influential predictors, ranked in descending order of importance, were uncertainty about the pandemic’s duration and risks, poor health, social isolation, conflicts at home, loss of daily structure, fear of infection, and restricted personal contact with close others.
Conclusions
AjD symptoms were most strongly linked to factors related to lack of control (e.g., uncertainty, loss of daily structure, fear of infection), as well as current poor health and reduced social connectedness. Interventions that enhance a sense of control through clear communication, help individuals re-establish daily routines, and strengthen social connectedness may mitigate AjD symptoms during future public health crises. Our findings also highlight the potential of machine learning approaches for identifying complex patterns across high-dimensional predictors of clinical symptoms, which may improve prediction accuracy in mental health research.
When the outcome Y is discrete rather than continuous, we refer to the problem of predicting Y as classification. In many ways, this is easier than predicting a continuous outcome since Y can only take a few values. Most of the methods we have covered so far can be adapted to handle discrete outcomes. One particular method, based on neural nets, is covered in Chapter 12.
In this study, we explore L1 and L2 speakers’ use of degree modifiers (DMs) aika/melko/ihan and quite/rather/fairly in a cross-linguistic setting, with academic Finnish and English as languages of interest. As a method, we apply a multivariate approach that considers the constructional features of the DMs. The statistical modelling showed reliably that, in both languages, L1 and L2 speakers made partially different choices when using the DMs. The model predicted the DM use of both languages well, although it explained the variation of the Finnish DMs better. In general, the English L2 use of the DMs was closer to English L1 use than was the case in Finnish, where the populations had a clearly different favourite among the three DM variants. The results suggest that the examined DM group is more fixed in academic Finnish, whereas in academic English the choice between the examined DM variants is more open.
Tree-based methods are widely used in insurance pricing due to their simple and accurate splitting rules. However, there is no guarantee that the resulting premiums avoid indirect discrimination when features recorded in the database are correlated with the protected variable under consideration. This paper shows that splitting rules in regression trees and random forests can be adapted in order to avoid indirect discrimination related to a binary protected variable like gender. The new procedure is illustrated on motor third-party liability insurance claim data.
This study incorporates a random forest (RF) approach to probe complex interactions and nonlinearity among predictors into an item response model with the goal of using a hybrid approach to outperform either an RF or explanatory item response model (EIRM) only in explaining item responses. In the specified model, called EIRM-RF, predicted values using RF are added as a predictor in EIRM to model the nonlinear and interaction effects of person- and item-level predictors in person-by-item response data, while accounting for random effects over persons and items. The results of the EIRM-RF are probed with interpretable machine learning (ML) methods, including feature importance measures, partial dependence plots, accumulated local effect plots, and the H-statistic. The EIRM-RF and the interpretable methods are illustrated using an empirical data set to explain differences in reading comprehension in digital versus paper mediums, and the results of EIRM-RF are compared with those of EIRM and RF to show empirical differences in modeling the effects of predictors and random effects among EIRM, RF, and EIRM-RF. In addition, simulation studies are conducted to compare model accuracy among the three models and to evaluate the performance of interpretable ML methods.
This chatper first introduces the kernel trick, which allows us to operate in the original lower-dimensional domain. We then discuss decision tree and ensemble methods for reducing data over-fitting.
This chapter covers regression and classification, where the goal is to estimate a quantity of interest (the response) from observed features. In regression, the response is a numerical variable. In classification, it belongs to a finite set of predetermined classes. We begin with a comprehensive description of linear regression and discuss how to leverage it to perform causal inference. Then, we explain under what conditions linear models tend to overfit or to generalize robustly to held-out data. Motivated by the threat of overfitting, we introduce regularization and ridge regression, and discuss sparse regression, where the goal is to fit a linear model that only depends on a small subset of the available features. Then, we introduce two popular linear models for binary and multiclass classification: Logistic and softmax regression. At this point, we turn our attention to nonlinear models. First, we present regression and classification trees and explain how to combine them via bagging, random forests, and boosting. Second, we explain how to train neural networks to perform regression and classification. Finally, we discuss how to evaluate classification models.
This chapter will cover new techniques beyond probing empirical data for data exploration. It will show you how to use a conditional inference tree (ctree) and random forest (cforest) to understand complex data interactions, pinpoint difficulties in research design, and discover data anomalies.The focus will be on techniques for resolving data and linguistic problems in preparation for statistical modelling
Classification and Regression Trees (CART), and their successors—bagging and random forests, are statistical learning tools that are receiving increasing attention. However, due to characteristics of censored data collection, standard CART algorithms are not immediately transferable to the context of survival analysis. Questions about the occurrence and timing of events arise throughout psychological and behavioral sciences, especially in longitudinal studies. The prediction power and other key features of tree-based methods are promising in studies where an event occurrence is the outcome of interest. This article reviews existing tree algorithms designed specifically for censored responses as well as recently developed survival ensemble methods, and introduces available computer software. Through simulations and a practical example, merits and limitations of these methods are discussed. Suggestions are provided for practical use.
Chapter 8 presents random forests for regression, which – at least in some situations – may outperform the least-squares-based regression methods. The chapter discusses bagging in the context of regression applications of random forests, the algorithm for splitting nodes in regression trees, and the variable importance metrics applicable to regression.
Chapter 10 covers the random forests algorithm for classification. Presented are also the impurity metrics applicable to splitting nodes in classification trees (Gini, entropy, and misclassification impurity), as well as permutation-based and impurity-based variable importance measures.
Chapter 17 describes the second real-life study, whose goal is the identification of multivariate biomarkers for liver cancer. This study implements parallel recursive feature elimination experiments coupled with random forests and support vector machines. Included are also considerations for rebalancing class proportions. Three multivariate biomarkers for liver cancer have been identified. The study has been performed in an R environment, and R scripts for all of its steps are provided.
Tree-based methods use methodologies that are radically different from those discussed in previous chapters. They are relatively easy to use and can be applied to a wide class of problems. As with many of the new machine learning methods, construction of a tree, or (in the random forest approach, trees) follows an algorithmic process. Single-tree methods occupy the first part this chapter. An important aspect of the methodology is the determining of error estimates. By building a large number of trees and using a voting process to make predictions, the random forests methodology that occupies the latter part of this chapter can often greatly improve on what can be achieved with a single tree. The methodology operates more as a black box, but with implementation details that are simpler to describe than for single- tree methods. In large sample classification problems, the methodology has often proved superior to other contenders.
Machine learning has recently entered the mortality literature in order to improve the forecasts of stochastic mortality models. This paper proposes to use two pure, tree-based machine learning models: random forests and gradient boosting, based on the differenced log-mortality rates to produce more accurate mortality forecasts. These forecasts are compared with forecasts from traditional, stochastic mortality models and with forecasts from random forests and gradient boosting variants of the stochastic models. The comparisons are based on the Model Confidence Set procedure. The results show that the pure, tree-based models significantly outperform all other models in the majority of cases considered. To address the lack of interpretability issue associated with machine learning models, we demonstrate how to extract information about the relationships uncovered by the tree-based models. For this purpose, we consider variable importance, partial dependence plots, and variable split conditions. Results from the in-sample fit suggest that tree-based models can be very useful tools for detecting patterns within and between variables that are not commonly identifiable with traditional methods.
In this study, prediction of aircraft Estimated Time of Arrival (ETA) is proposed using machine learning algorithms. Accurate prediction of ETA is important for management of delay and air traffic flow, runway assignment, gate assignment, collaborative decision making (CDM), coordination of ground personnel and equipment, and optimisation of arrival sequence etc. Machine learning is able to learn from experience and make predictions with weak assumptions or no assumptions at all. In the proposed approach, general flight information, trajectory data and weather data were obtained from different sources in various formats. Raw data were converted to tidy data and inserted into a relational database. To obtain the features for training the machine learning models, the data were explored, cleaned and transformed into convenient features. New features were also derived from the available data. Random forests and deep neural networks were used to train the machine learning models. Both models can predict the ETA with a mean absolute error (MAE) less than 6min after departure, and less than 3min after terminal manoeuvring area (TMA) entrance. Additionally, a web application was developed to dynamically predict the ETA using proposed models.
We study the scaling limit of a random forest with prescribed degree sequence in the regime that the largest tree consists of all but a vanishing fraction of nodes. We give a description of the limit of the forest consisting of the small trees, by relating a plane forest to a marked cyclic forest and its corresponding skip-free walk.
The panpipe is a musical instrument composed of end-blown tubes of different lengths tied together. They can be traced back to the Neolithic, and they have been found at prehistoric sites in China, Europe and South America. Panpipes display substantial variation in space and time across functional and aesthetic dimensions. Finding similarities in panpipes that belong to distant human groups poses a challenge to cultural evolution: while some have claimed that their relative simplicity speaks for independent inventions, others argue that strong similarities of specific features in panpipes from Asia, Oceania and South America suggest long-distance diffusion events. We examined 20 features of a worldwide sample of 401 panpipes and analysed statistically whether instrument features can successfully be used to determine provenance. The model predictions suggest that panpipes are reliable provenance markers, but we found an unusual classification error in which Melanesian panpipes are predicted as originating in South America. Although this pattern may be signalling a diffusion event, other factors such as convergence and preservation biases may play a role. Our analyses show the potential of cultural evolution research on music that incorporates material evidence, which in this study includes both archaeological and ethnographic samples preserved in museum collections.
Previous models suggest biological and behavioral continua among healthy individuals (HC), at-risk condition, and full-blown schizophrenia (SCZ). Part of these continua may be captured by schizotypy, which shares subclinical traits and biological phenotypes with SCZ, including thalamic structural abnormalities. In this regard, previous findings have suggested that multivariate volumetric patterns of individual thalamic nuclei discriminate HC from SCZ. These results were obtained using machine learning, which allows case–control classification at the single-subject level. However, machine learning accuracy is usually unsatisfactory possibly due to phenotype heterogeneity. Indeed, a source of misclassification may be related to thalamic structural characteristics of those HC with high schizotypy, which may resemble structural abnormalities of SCZ. We hypothesized that thalamic structural heterogeneity is related to schizotypy, such that high schizotypal burden would implicate misclassification of those HC whose thalamic patterns resemble SCZ abnormalities.
Methods
Following a previous report, we used Random Forests to predict diagnosis in a case–control sample (SCZ = 131, HC = 255) based on thalamic nuclei gray matter volumes estimates. Then, we investigated whether the likelihood to be classified as SCZ (π-SCZ) was associated with schizotypy in 174 HC, evaluated with the Schizotypal Personality Questionnaire.
Results
Prediction accuracy was 72.5%. Misclassified HC had higher positive schizotypy scores, which were correlated with π-SCZ. Results were specific to thalamic rather than whole-brain structural features.
Conclusions
These findings strengthen the relevance of thalamic structural abnormalities to SCZ and suggest that multivariate thalamic patterns are correlates of the continuum between schizotypy in HC and the full-blown disease.