Chapter 7 covers models with categorical endogenous variables. It examines the consequences of treating such variables as continuous and how to modify SEMs to take account of categorical variables. It begins with single equation regression-like models for binary, ordinal, and count variables and builds to multiequation models. It includes a polychoric correlation approach, models with exogenous observed variables, the treatment of missing values, and alternative modeling approaches for categorical variables.

This chapter provides a comprehensive introduction to supervised learning techniques for classification problems. It begins with logistic regression for binary classification, explaining the sigmoid function and gradient ascent optimization. The chapter then covers softmax regression for multi-class problems, followed by k-nearest neighbors (kNN) as an intuitive distance-based classifier.

Decision trees are explored in detail, including entropy, information gain, and the ID3 algorithm, along with derived decision rules and association rules. Random forests are presented as an ensemble method that addresses overfitting by combining multiple decision trees.

The chapter covers Naive Bayes classification based on Bayes’ theorem, despite its "naive" independence assumption. Finally, Support Vector Machines (SVMs) are introduced for both linear and non-linear classification using maximum margin hyperplanes.

Each technique includes hands-on R programming examples with real datasets, practical applications, and exercises to reinforce learning concepts.

This chapter explores supervised learning techniques where algorithms learn from labeled training data to make predictions. It begins with logistic regression for binary classification problems, using the sigmoid function to output probabilities between 0 and 1. Softmax regression extends this to multi-class problems. The chapter covers k-nearest neighbors (kNN), which classifies data points based on their similarity to training examples. Decision trees use entropy and information gain to create interpretable classification rules, while random forests combine multiple decision trees to reduce overfitting through ensemble methods. Naive Bayes applies Bayes’ theorem with independence assumptions for probabilistic classification, particularly effective for text classification. Finally, support vector machines (SVM) find optimal decision boundaries by maximizing margins between classes. Each technique is demonstrated through hands-on Python examples using real datasets, showing practical applications in various domains from healthcare to finance.

Regression and classification are closely related, as shown in this chapter, which discusses methods used to map a linear regression function into a probablity function by either logistic function (for binary classification) or softmax function (for multi-class classification). According to this probablity function, an unlabeled sample can be assigned to one of the classes. The optimal model parameters in this method can be obtained based on the training set so that either the likelihood or the posterior probability of these parameters are maximized.

This chapter focuses on the core concepts of optimization theory and its application in data science and AI. It begins with a review of differentiable functions of several variables, including the gradient and Hessian matrices, and key results like the Chain Rule and the Mean Value Theorem. The chapter then introduces optimality conditions for unconstrained optimization, explaining first-order and second-order conditions, and the role of convexity in ensuring global optimality. A detailed discussion of the gradient descent algorithm is provided, including its convergence analysis under different assumptions. The chapter concludes with an application to logistic regression, demonstrating how gradient descent is used to optimize the cross-entropy loss function in a supervised learning context. Practical Python examples are integrated throughout to illustrate the theoretical concepts.

This chapter covers regression and classification, where the goal is to estimate a quantity of interest (the response) from observed features. In regression, the response is a numerical variable. In classification, it belongs to a finite set of predetermined classes. We begin with a comprehensive description of linear regression and discuss how to leverage it to perform causal inference. Then, we explain under what conditions linear models tend to overfit or to generalize robustly to held-out data. Motivated by the threat of overfitting, we introduce regularization and ridge regression, and discuss sparse regression, where the goal is to fit a linear model that only depends on a small subset of the available features. Then, we introduce two popular linear models for binary and multiclass classification: Logistic and softmax regression. At this point, we turn our attention to nonlinear models. First, we present regression and classification trees and explain how to combine them via bagging, random forests, and boosting. Second, we explain how to train neural networks to perform regression and classification. Finally, we discuss how to evaluate classification models.

How do I conduct a mixed effects logistic regression of a linguistic variable?This chapter will illustrate the procedures for performing statistical modelling using mixed effects logistic regression with the lme4 package in R. It will review the steps for conducting analyses, for finding the best model for the feature under study, and what to do with it when you find it.

A rich and important area for the applications of linear algebra is machine learning. In machine learning, one aims to achieve optimized or learned understanding of various kinds of real-world phenomena from data collected or observed, without real comprehension of the functioning mechanisms of such phenomena. These functioning mechanisms are often impossible or unpractical to grasp anyway. In this chapter, we present several introductory and fundamental problems in supervised machine learning including linear regression, data classification, and logistic regression and the mathematical and computational methods associated.

In this chapter, new computational models will focus on whether environmental health texts are suitable for parents rather than the general public. Logistic regression models will identify linguistic features that are important contributors to the prediction of the suitability of environmental health materials for parents and caregivers of young children, who are more likely to be affected by environmental health risks such as water pollution, excessive sun exposure, and radiation in natural and indoor environments.

This chapter describes how to characterize data and the distribution of data. We will also describe how the shape of the normal distribution enables hypothesis testing. In the section on regression, we look at how two variables or ways of measuring data are related to each other. We will use simple linear regression as an introduction to multiple regression, the technique used in the development of a number of traditional readability measures. A more sophisticated form of regression is called logistic regression is also discussed, which will be applied in the case studies of Chapters 4 to 6.

This chapter examines the conceptualization and measurement of contact phenomena in the context of bilingualism across various languages. The goal of the chapter is to account for various phonetic contact phenomena in sociolinguistic analysis, as well as providing context for elaborating on quantitative methodologies in sociophonetic contact linguistics. More specifically, the chapter provides a detailed account of global phenomena in modern natural speech contexts, as well as an up-to-date examination of quantitative methods in the field of sociolinguistics. The first section provides a background of theoretical concepts important to the understanding of sociophonetic contact in the formation of sound systems. The following sections focus on several key social factors that play a major part in the sociolinguistic approach to bilingual phonetics and phonology, including language dominance and age of acquisition at the segmental and the suprasegmental levels, as well as topics of language attitudes and perception, and typical quantitative methods used in sociolinguistics.