Essentials of Pattern Recognition An Accessible Approach

There is no silver bullet: no model can fit all data. Hence, special data requires special algorithms. In this chapter, we deal with two types of special data: sparse data and sequences that can be aligned to each other. We will not dive deep into sparsity learning, which is very complex. Rather, we introduce key concepts: sparsity inducing loss functions, dictionary learning, and what exactly the word sparsity means. For the second part in this chapter, we introduce dynamic time warping (DTW), which deals with sequences that can be aligned with each other (but there are sequences that cannot be aligned, which we will discuss in the next chapter). We use our old tricks: ideas, visualizations, formalizations, to reach the DTW solution. The key idea behind its success is divide-and-conquer and the key technology is dynamic programming.