To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
This chapter provides an overview of the types of inference problems we address and the different approaches to solving them. We focus on risky inference: drawing conclusions, learning and making predictions in situations where certainty is impossible. Predicting a response from one or more predictors using past data is called supervised learning. When the response is continuous, the task is regression; when it is categorical, the task is classification. In unsupervised learning, there is no response variable. Instead, the goal is to find patterns or structure in data, as in density estimation, clustering and dimensionality reduction. In both supervised and unsupervised contexts, overfitting occurs when we model data in excessive detail and fail to distinguish systematic patterns from noise; underfitting occurs when our models are too simple to capture systematic patterns. Probability is a key tool for tackling risky inference, with frequentist and Bayesian interpretations motivating distinct approaches. Finally, large neural networks have proven remarkably effective in both supervised and unsupervised tasks, often avoiding overfitting despite containing billions of parameters.
This paper presents a deep learning-based approach to automatically classify the rust level of screws using ResNet-18 and MobileNetV3 convolutional neural networks. A controlled salt-spray chamber was used to simulate corrosion on metal screws over 0h, 48h, 96h, and 168h of exposure. Images were processed with a circle-detection algorithm to extract individual screws, followed by data augmentation and training. The final models achieved a classification accuracy greater than 94% on the validation set.
Modern aviation supports an ever-broader range of civil and military missions, and the airframes designed for these missions must satisfy stringent safety and performance requirements. The takeoff and landing phases are the most accident-prone portions of a flight despite representing only a short interval of the total block time, which makes the accurate prediction of takeoff speed a safety-relevant problem. A previous machine learning study addressed the takeoff-speed prediction problem of the Boeing 737-300 with classical regressors using pressure altitude, outside air temperature, gross weight and flap angle as the predictors. In the present work, the same regression problem is revisited under the deep learning paradigm. Four neural architectures are trained on an identical pre-processing pipeline and train-validation partition, namely a multilayer perceptron, a one-dimensional convolutional network, a long short-term memory network and a wide-and-deep architecture incorporating multi-head self-attention. Among the four candidates, the long short-term memory network attains the lowest root mean square error and mean square error on the unseen test file and is subsequently subjected to Bayesian hyperparameter optimisation through the Keras Tuner library. The predicted and the measured takeoff speeds are reported side by side for the first time in the deep learning literature for this airframe, and the simulation results indicate that the developed networks constitute an effective alternative tool for takeoff-speed prediction.
In this chapter, we briefly cover a few other topics related to regression. Each topic is the subject of entire textbooks. Our goal is to give a very concise introduction to each topic. The topics include random effects and empirical Bayes, neural nets and deep learning, survival analysis, graphical models, and time series.
Early detection of spider mite emergence is highly challenging due to the lack of visible symptoms and subtle physiological changes. To address the rapid monitoring of pest mite damage, this study provided a method based on hyperspectral imaging combined with a joint spatial-spectral attention (SSA) mechanism in a deep convolutional neural network (DCNN) for automated detection of pest mite infections. Leaves infested with varying degrees of spider mites Tetranychus urticae Koch (Acari: Tetranychidae) were captured daily to obtain multi-band spectral information of hyperspectral images. After data preprocessing, the joint SSA mechanism was applied to weigh and optimise the spatial and spectral features of the images, focusing specifically on infested regions. Furthermore, the super-pixel principal component analysis method (SPCA) dimensionality reduction method was applied to reduce model complexity and enhance classification accuracy. The spatial attention module automatically adjusted the weights of critical regions in the image, ensuring the network concentrated on the infected areas, while the spectral attention module improved sensitivity to the unique spectral features of infected regions. The proposed method can significantly enhance the accuracy of identifying mite infected areas, particularly in detecting mild infections of the leaves. Compared with traditional approaches, our proposed SPCA + DCNN + SSA model achieved notable improvements in classification accuracy and robustness, yielding the highest OA (up to 99.1% for specific severity levels) and Kappa coefficient (0.989). Importantly, it drastically reduced misclassification in the highly challenging mild infection stages.
In this study, we analysed differences in the infant gut microbiome between breastfed and formula-fed infants using novel machine learning techniques. Breast milk, rich in bioactive agents, supports microbiota composition and immune development, while formulas aim to replicate its nutritional profile. We applied a methodology combining the DADA2 pipeline for 16S rRNA sequencing with the Recursive Ensemble Feature Selection (REFS) algorithm for biomarker discovery. We analysed three publicly available 16S rRNA datasets: PRJNA633365 (70 stool samples from China), PRJDB7295 (40 stool samples from the Philippines), and PRJNA562650 (40 stool samples from China). The discovery dataset (PRJNA633365) revealed 16 significant taxa out of 1,227, validated across the other two datasets. Next, we compared REFS performance with another feature selection algorithm, SelectKBest. Finally, we conducted a literature review to explore links between identified taxa and medical conditions. Additionally, we used MicrobiomeAnalyst to examine associations with diseases, diet, and lifestyle. Our results show differences in the bacterial composition between breastfed and formula-fed infants, and these findings were validated in two independent datasets. Future research should explore the functional roles of these taxa and consider regional and dietary variability to enhance understanding of microbiome dynamics and long-term health outcomes.
Archaeologists have demonstrated the value of deep learning models for detecting archaeological objects in lidar data. As landscape-level projects become the norm, archaeological data derived from deep learning predictions can be integrated into these initiatives through coupled natural-cultural landscapes planning. However, the paucity of archaeological training datasets limits the application of deep learning models to relatively common and well-documented object classes. Using procedurally generated training datasets may be one approach to overcome this bottleneck. To test the efficacy of procedural generation for developing deep learning training data, we trained models to detect a novel object class (hypothesized historic tar kilns) in the Kisatchie National Forest in Louisiana. We developed two procedural generation approaches to embed simulated archaeological objects in a lidar-derived DEM and used these datasets to train deep learning (Mask R-CNN) models. We then evaluated model predictions within lidar-derived visualizations and during field survey. Our trained models detected targets with high recall but low precision. Field investigation suggested that the objects were not tar kilns but a different historic feature class. This study suggests that models trained on simulated objects are a useful addition to lidar analysis tool kits and can be directly integrated into archaeological field investigation workflows.
Federated Learning is a novel method of training machine learning models, pioneered by Google, aimed for use on smartphones. In contrast to traditional machine learning, where data is centralised and brought to the model, Federated Learning involves the algorithm being brought to the data, ensuring privacy is preserved. This paper will demonstrate how insurance companies in a market could use this technique to build a claims frequency neural network prediction model collectively by combining and using all of their customer data, without actually sharing or compromising any sensitive information with each other. A simulated car insurance market with 10 players was created using the freMTPL2freq dataset. It was found that if all insurers were permitted to share their confidential data with each other, they could collectively build a model that achieved 5.57% of exposure weighted Poisson Deviance Explained (% PDE) on an unseen sample. However, if they are not permitted to share their customer data, none of them can achieve more than 3.82% exposure weighted PDE on the same unseen sample. With Federated Learning, they can retain all of their customer data privately and construct a model that achieves a similar level of accuracy to that achieved by centralising all the data for model training, reaching 5.34% exposure weighted PDE on the same unseen sample.
The manual identification of ancient agricultural terraces is time-consuming and subjective, limiting large-scale archaeological landscape documentation. This study applies deep learning to detect ancient terraces in the Bozburun Peninsula, southwestern Turkey, a historically significant Hellenistic landscape. Four U-Net–based architectures were implemented—early, intermediate, and late fusion, along with an RGB-only baseline—integrating high-resolution aerial imagery (30 cm) and digital elevation models (DEMs) across 193 km2. Sixteen manually digitized areas (37.8 ha) produced 256 training patches (512 × 512 px). The early fusion model that combined spectral and topographic data achieved the best performance (IoU = 0.754; accuracy = 85.9%). Monte Carlo evaluation confirmed its robustness. Spatial analysis showed that 89.8% of detected terraces lie below 300 m elevation, mainly on 10°–20° slopes with north-northwest orientation, in agreement with previous archaeological observations. Compared with expert digitization, the model yielded higher precision (87.4% vs. 79.3%), while experts achieved higher recall (94.3% vs. 76.6%). Applied to the full peninsula, the model mapped 2,517 ha of terraces. Validation using an existing archaeological dataset (Demirciler 2014) enabled direct comparison between automated and expert-based interpretations. The results indicate the potential of deep learning for terrace detection in Mediterranean landscapes and outline a methodological framework for documenting threatened cultural heritage.
Chapter 1 introduces basic terminology. Terms such as artificial intelligence, data, algorithm, machine learning, neural networks, deep learning, large language models, generative AI and symbolic AI are presented to develop a sense of what AI is, how it has evolved, and what it does. This chapter also introduces some of the major conceptual disagreements in the field. Different ideas about how to develop AI in the best way drive disagreements, as well as philosophical differences over what intelligence means and whether machines can develop human-like intelligence.
Emphasizing how and why machine learning algorithms work, this introductory textbook bridges the gap between the theoretical foundations of machine learning and its practical algorithmic and code-level implementation. Over 85 thorough worked examples, in both Matlab and Python, demonstrate how algorithms are implemented and applied whilst illustrating the end result. Over 75 end-of-chapter problems empower students to develop their own code to implement these algorithms, equipping them with hands-on experience. Matlab coding examples demonstrate how a mathematical idea is converted from equations to code, and provide a jumping off point for students, supported by in-depth coverage of essential mathematics including multivariable calculus, linear algebra, probability and statistics, numerical methods, and optimization. Accompanied online by instructor lecture slides, downloadable Python code and additional appendices, this is an excellent introduction to machine learning for senior undergraduate and graduate students in Engineering and Computer Science.
With the widespread application of smart antennas in 5G communication and radar detection, adaptive beamforming technology based on deep learning has become a research focus for improving the anti-interference performance of antenna arrays due to its powerful nonlinear modeling capability. It can transform the beamforming problem into a neural network regression problem, enabling the model to rapidly output an approximately optimal beamforming weight vector without prior information. Aiming at the issues of poor adaptability to dynamic interference and high computational complexity of traditional algorithms, this paper proposes IRDSNet, a novel adaptive beamforming algorithm based on Inception-ResNet-dual-pool Squeeze-and-Excitation Network (DP-SENet), to optimize the performance of uniform circular array antennas. IRDSNet integrates the Inception structure, depthwise separable convolution, and Ghost convolution to construct a multi-scale feature extraction module, enhancing the model’s feature extraction capabilities while maintaining a low parameter count. By introducing an improved DP-SENet, the model’s ability to focus on key features is enhanced, while the incorporation of residual modules optimizes feature transmission efficiency. Simulation results demonstrate that the IRDSNet algorithm achieves a null depth exceeding −90 dB at various interference angles, with an output Signal-to-Interference-plus-Noise Ratio (SINR) consistently above 23 dB and a short inference time, demonstrating excellent interference suppression performance.
The chapter begins with a discussion on standard mechanisms for training spiking neural networks ranging from – (a) unsupervised spike-timing-dependent plasticity, (b) backpropagation through time (BPTT) using surrogate gradient techniques, and (c) conversion techniques from conventional analog non-spiking networks. Subsequently, various local learning algorithms with different degrees of locality are discussed that have the potential to replace computationally expensive global learning algorithms such as BPTT. The chapter concludes with pointers to several emerging research directions in the neuromorphic algorithms domain ranging from stochastic computing, lifelong learning, and dynamical system-based approaches, among others. Finally, we also underscore the need for looking at hybrid neuromorphic algorithm design combining principles of conventional deep learning along with forging stronger connections with computational neuroscience.
The chapter introduces fundamental principles of deep learning. We discuss supervised learning of feedforward neural networks by considering a binary classification problem. Gradient descent techniques and backpropagation learning algorithms are introduced as means of training neural networks. The impact of neuron activations and convolutional and residual network architectures on the learning performance are discussed. Finally, regularization techniques such as batch normalization and dropout are introduced for improving the accuracy of trained models. The chapter is essential to connect advances in conventional deep learning algorithms to neuromorphic concepts.
The paper introduces a deep‐learning model fine‐tuned for detecting authoritarian discourse in political speeches. Set up as a regression problem with weak supervision logic, the model is trained for the task of classification of segments of text for being/not being associated with authoritarian discourse. Rather than trying to define what an authoritarian discourse is, the model builds on the assumption that authoritarian leaders inherently define it. In other words, authoritarian leaders talk like authoritarians. When combined with the discourse defined by democratic leaders, the model learns the instances that are more often associated with authoritarians on the one hand and democrats on the other. The paper discusses several evaluation tests using the model and advocates for its usefulness in a broad range of research problems. It presents a new methodology for studying latent political concepts and positions as an alternative to more traditional research strategies.
Pater's (2019) target article builds a persuasive case for establishing stronger ties between theoretical linguistics and connectionism (deep learning). This commentary extends his arguments to semantics, focusing in particular on issues of learning, compositionality, and lexical meaning.
Joe Pater's (2019) target article calls for greater interaction between neural network research and linguistics. I expand on this call and show how such interaction can benefit both fields. Linguists can contribute to research on neural networks for language technologies by clearly delineating the linguistic capabilities that can be expected of such systems, and by constructing controlled experimental paradigms that can determine whether those desiderata have been met. In the other direction, neural networks can benefit the scientific study of language by providing infrastructure for modeling human sentence processing and for evaluating the necessity of particular innate constraints on language acquisition.
The birthdate of both generative linguistics and neural networks can be taken as 1957, the year of the publication of foundational work by both Noam Chomsky and Frank Rosenblatt. This article traces the development of these two approaches to cognitive science, from their largely autonomous early development in the first thirty years, through their collision in the 1980s around the past-tense debate (Rumelhart & McClelland 1986, Pinker & Prince 1988) and their integration in much subsequent work up to the present. Although this integration has produced a considerable body of results, the continued general gulf between these two lines of research is likely impeding progress in both: on learning in generative linguistics, and on the representation of language in neural modeling. The article concludes with a brief argument that generative linguistics is unlikely to fulfill its promise of accounting for language learning if it continues to maintain its distance from neural and statistical approaches to learning.
This research paper addresses the hypothesis that sequence-based long short-term memory (LSTM) architectures improve the prediction of the next DO (days open) relative to a feed-forward multi-layer perceptron and a Cox model under strictly temporally valid predictors. Modern dairy farming can heavily benefit from optimising ‘days open’ for profitability and animal welfare. Machine learning can forecast this metric, improving farm management, disease prevention and culling decisions. This study used a dataset of 16,472 breeding records. The study compared the performance of feed-forward neural networks and two types of recurrent neural networks (RNNs). The results showed that LSTM most accurately forecasted the next ‘days open’. This demonstrates that RNN models, due to their ability to capture temporal patterns in the data, significantly outperform feed-forward and traditional statistical methods in terms of mean absolute error and concordance.
This chapter first introduces a simple two-layer perceptron network based on some straight forward learning rule. This perceptron network can be used as a linear classifier capable of multiclass classification if the classes are linearly separable, which can be further generalized for nonlinear classification when the kernel method is introduced into the algorithm. The main algorithm discussed in this chapter is the multi-layer (3 or more) back propagation network which is a supervised method most widely used for classification, and also serves as one of the building blocks of the much more powerful deep learning method and other artificial intelligence methods. Based on the labeled sample in the training set, the weights of the back propagation network are sequentially modified in the training process in such a way that the error, the difference between the actual out and the desired outputs, the ground truth labeling of its input, is reduced by the gradient descent method. Based on the same training process, this network can be modified to serve as an autoencoder for dimensionality reduction, similar to what the PCA can do.