In this part we show how density-ratio estimation methods can be used for solving various machine learning problems.

In the context of importance sampling (Fishman, 1996), where the expectation over one distribution is computed by the importance-weighted expectation over another distribution, density ratios play an essential role. In Chapter 9, the importance sampling technique is applied to non-stationarity/domain adaptation in the semi-supervised learning setup (Shimodaira, 2000; Zadrozny, 2004; Sugiyama and Müller, 2005; Storkey and Sugiyama, 2007; Sugiyama et al., 2007; Quiñonero- Candela et al., 2009; Sugiyama and Kawanabe, 2011). It is also shown that the same importance-weighting idea can be used for solving multi-task learning (Bickel et al., 2008).

Another major usage of density ratios is distribution comparisons. In Chapter 10, two methods of distribution comparison based on density-ratio estimation are described: inlier-base outlier detection, where distributions are compared in a pointwise manner (Smola et al., 2009; Hido et al., 2011), and two-sample tests, where the overall difference between distributions is compared within the framework of hypothesis testing (Sugiyama et al., 2011c).

In Chapter 11 we show that density-ratio methods allow one to accurately estimate mutual information (Suzuki et al., 2008, 2009a). Mutual information is a key quantity in information theory (Cover and Thomas, 2006), and it can be used for detecting statistical independence between random variables.

In this chapter we explain the usage of density-ratio estimation for comparing probability distributions.

In Section 10.1, the pointwise difference of two densities is considered for evaluating whether a sample drawn from one distribution is “typical” in the other distribution. This problem is referred to as inliner-based outlier detection, where the degree of outlyingness of samples in an evaluation dataset is examined based on another dataset that consists only of inlier samples (Smola et al., 2009; Hido et al., 2011).

In Section 10.2, the overall difference of two densities is considered, which basically corresponds to estimating a divergence between two densities. The estimated divergence can be used for a two-sample test, which is aimed at judging whether two distributions are the same within the framework of hypothesis testing (Sugiyama et al., 2011c).

Inlier-Based Outlier Detection

In this section we show how density-ratio methods can be used for inlier-based outlier detection (Smola et al., 2009; Hido et al., 2011). After an introduction in Section 10.1.1, we formulate the problem of inlier-based outlier detection and show how it can be solved via density-ratio estimation in Section 10.1.2. Various native outlier-detection methods are reviewed in Section 10.1.3, and experimental performance is compared in Section 10.1.4. Finally, the section is concluded in Section 10.1.5.

Introduction

The goal of outlier detection (a.k.a. anomaly detection, novelty detection, or one-class classification) is to find uncommon instances (“outliers”) in a given dataset.

The approaches of direct density-ratio estimation explained in the previous chapters were shown to be promising in experiments with naive kernel density estimation in experiments. However, these methods still perform rather poorly when the dimensionality of the data domain is high.

The purpose of this chapter is to introduce ideas for mitigating this weakness, following Sugiyama et al. (2010a, 2011b). A basic assumption behind the approaches explained here is that the difference between the two distributions in the density ratio (i.e., the distributions corresponding to the numerator and denominator of the density ratio) does not spread over the entire data domain, but is confined in a low-dimensional subspace – which we refer to as the heterodistributional subspace. Once the heterodistributional subspace can be identified, the density ratio is estimated only within this subspace. This will lead to more stable and reliable estimations of density ratios. Such an approach is called direct density-ratio estimation with dimensionality reduction (D3; pronounced “D-cube”).

In this chapter, two approaches to D3 are described. In Section 8.1, a heuristic method based on discriminant analysis is explained. This method is shown to be computationally very efficient, and thus is very practical. On the other hand, in Section 8.2, a more theory-oriented approach based on divergence maximization is introduced. This method is justifiable under general settings, and thus it has a wider applicability. Numerical examples are shown in Section 8.3, and the chapter is concluded in Section 8.4.

Estimating probability distributions is widely viewed as a central question in machine learning. The whole enterprise of probabilistic modeling using probabilistic graphical models is generally addressed by learning marginal and conditional probability distributions. Classification and regression – starting with Fisher's fundamental contributions – are similarly viewed as problems of estimating conditional densities.

The present book introduces an exciting alternative perspective – namely, that virtually all problems in machine learning can be formulated and solved as problems of estimating density ratios – the ratios of two probability densities. This book provides a comprehensive review of the elegant line of research undertaken by the authors and their collaborators over the last decade. It reviews existing work on density-ratio estimation and derives a variety of algorithms for directly estimating density ratios. It then shows how these novel algorithms can address not only standard machine learning problems – such as classification, regression, and feature selection – but also a variety of other important problems such as learning under a covariate shift, multi-task learning, outlier detection, sufficient dimensionality reduction, and independent component analysis.

At each point this book carefully defines the problems at hand, reviews existing work, derives novel methods, and reports on numerical experiments that validate the effectiveness and superiority of the new methods. A particularly impressive aspect of the work is that implementations of most of the methods are available for download from the authors’ web pages.

In this chapter we describe a framework of density-ratio estimation by density-ratio fitting under the Bregman divergence (Bregman, 1967). This framework is a natural extension of the least-squares approach described in Chapter 6, and it includes various existing approaches as special cases (Sugiyama et al., 2011a).

In Section 7.1 we first describe the framework of density-ratio fitting under the Bregman divergence. Then, in Section 7.2, we show that various existing approaches can be accommodated in this framework, such as kernel mean matching (see Section 3.3), logistic regression (see Section 4.2), Kullback–Leibler importance estimation procedure (see Section 5.1), and least-squares importance fitting (see Section 6.1).We then show other views of the density-ratio fitting framework in Section 7.3. Furthermore, in Section 7.4, a robust density-ratio estimator is derived as an instance of the density-ratio fitting approach based on Basu's power divergence (Basu et al., 1998). The chapter is concluded in Section 7.5.

Basic Framework

A basic idea of density-ratio fitting is to fit a density ratio model r(x) to the true density-ratio function r*(x) under some divergence (Figure 7.1). At a glance, this density-ratio fitting problem may look equivalent to the regression problem, which is aimed at learning a real-valued function [see Section 1.1.1 and Figure 1.1(a)]. However, density-ratio fitting is essentially different from regression because samples of the true density-ratio function are not available. Here we employ the Bregman (BR) divergence (Bregman, 1967) for measuring the discrepancy between the true density-ratio function and the density-ratio model.

Machine learning is aimed at developing systems that learn. The mathematical foundation of machine learning and its real-world applications have been extensively explored in the last decades. Various tasks of machine learning, such as regression and classification, typically can be solved by estimating probability distributions behind data. However, estimating probability distributions is one of the most difficult problems in statistical data analysis, and thus solving machine learning tasks without going through distribution estimation is a key challenge in modern machine learning.

So far, various algorithms have been developed that do not involve distribution estimation but solve target machine learning tasks directly. The support vector machine is a successful example that follows this line – it does not estimate data generating distributions but directly obtains the class-decision boundary that is sufficient for classification. However, developing such an excellent algorithm for each of the machine learning tasks could be highly costly and difficult.

To overcome these limitations of current machine learning research, we introduce and develop a novel paradigm called density-ratio estimation – instead of probability distributions, the ratio of probability densities is estimated for statistical data processing. The density-ratio approach covers various machine learning tasks, for example, non-stationarity adaptation, multi-task learning, outlier detection, two-sample tests, feature selection, dimensionality reduction, independent component analysis, causal inference, conditional density estimation, and probabilitic classification. Thus, density-ratio estimation is a versatile tool for machine learning. This book is aimed at introducing the mathematical foundation, practical algorithms, and applications of density-ratio estimation.