While previous chapters discussed deep learning recommender systems from a theoretical and algorithmic perspective, this chapter shifts focus to the engineering platform that supports their implementation. Recommender systems are divided into two key components: data and model. The data aspect involves the engineering of the data pipeline, while the model aspect is split between offline training and online serving. This chapter is structured into three parts: (1) the data pipeline framework and big data platform technologies; (2) popular platforms for offline training of recommendation models like Spark MLlib, TensorFlow, and PyTorch; and (3) online deployment and serving of deep learning recommendation models. Additionally, the chapter covers the trade-offs between engineering execution and theoretical considerations, offering insights into how algorithm engineers can balance these aspects in practice.

Chapter 12 is the conclusion. It presents a discussion of how the components of performance evaluation for learning algorithms discussed throughout the book unify into an overall framework for in-laboratory evaluation. This is followed by a discussion of how to move from a laboratory setting to a deployment setting based on the material covered in the last part of the book. We then discuss the potential social consequences of machine learning technology deployment together with their causes, and advocate for the consideration of these consequences as part of the evaluation framework. We follow this discussion with a few concluding remarks.

Chapter 4 reviews frequently used machine learning evaluation procedures. In particular, it presents popular evaluation metrics for binary and multi-class classification (e.g., accuracy, precision/recall, ROC analysis), regression analysis (e.g., mean squared error, root mean squared error, R-squared error), clustering (e.g., Davies–Bouldin Index). It then reviews popular resampling approaches (e.g.,holdout, cross-validation) and statistical tests (e.g., the t-test and the sign test). It concludes with an explanation of why it is important to go beyond these well-known methods in order to achieve reliable evaluation results in all cases.

Chapter 6 addresses the problem of error estimation and resampling in both a theoretical and practical manner. The holdout method is reviewed and cast into the bias/variance framework. Simple resampling approaches such as cross-validation are also reviewed and important variations such as stratified cross-validation and leave-one-out are introduced. Multiple resampling approaches such as bootstrapping, randomization, and multiple trials of simple resampling approaches are then introduced and discussed.

Chapter 2 reviews the principles of statistics that are necessary for the discussion of machine learning evaluation methods, especially the statical analysis discussion of Chapter 7. In particular, it reviews the notions of random variables, distributions, confidence intervals, and hypothesis testing.

In Chapter 10, the book turns to practical considerations. In particular, it surveys the software engineering discipline with its rigorous software testing methods, and asks how these techniques can be adapted to the subfield of machine learning. The adaptation is not straightforward, as machine learning algorithms behave in non-deterministic ways aggravated by data, algorithm, and platform imperfections. These issues are discussed and some of the steps taken to handle them are reviewed. The chapter then turns to the practice of online testing and addresses the ethics of machine learning deployment. The chapter concludes with a discussion of current industry practice along with suggestions on how to improve the safety of industrial deployment in the future.