Skip to main content Accessibility help
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Pattern Recognition and Machine Learning
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

2327 results in Pattern Recognition and Machine Learning

Page 52 of 117

  • 4 - Information-Theoretic Bounds on Sketching

  • Edited by Miguel R. D. Rodrigues, University College London, Yonina C. Eldar, Weizmann Institute of Science, Israel
  • Book:
    Information-Theoretic Methods in Data Science
    Published online:
    22 March 2021
    Print publication:
    08 April 2021, pp 104-133

  • Summary

    Approximate computation methods with provable performance guarantees are becoming important and relevant tools in practice. In this chapter we focus on sketching methods designed to reduce data dimensionality in computationally intensive tasks. Sketching can often provide better space, time, and communication complexity trade-offs by sacrificing minimal accuracy. This chapter discusses the role of information theory in sketching methods for solving large-scale statistical estimation and optimization problems. We investigate fundamental lower bounds on the performance of sketching. By exploring these lower bounds, we obtain interesting trade-offs in computation and accuracy. We employ Fano’s inequality and metric entropy to understand fundamental lower bounds on the accuracy of sketching, which is parallel to the information-theoretic techniques used in statistical minimax theory.

  • 5 - Sample Complexity Bounds for Dictionary Learning from Vector- and Tensor-Valued Data

  • Edited by Miguel R. D. Rodrigues, University College London, Yonina C. Eldar, Weizmann Institute of Science, Israel
  • Book:
    Information-Theoretic Methods in Data Science
    Published online:
    22 March 2021
    Print publication:
    08 April 2021, pp 134-162

  • Summary

    Dictionary learning has emerged as a powerful method for data-driven extraction of features from data. The initial focus was from an algorithmic perspective, but recently there has been increasing interest in the theoretical underpinnings. These rely on information-theoretic analytic tools and help us understand the fundamental limitations of dictionary-learning algorithms. We focus on theoretical aspects and summarize results on dictionary learning from vector- and tensor-valued data. Results are stated in terms of lower and upper bounds on sample complexity of dictionary learning, defined as the number of samples needed to identify or reconstruct the true dictionary underlying data from noiseless or noisy samples, respectively. Many analytic tools that help yield these results come from information theory, including restating the dictionary-learning problem as a channel-coding problem and connecting analysis of minimax risk in statistical estimation to Fano’s inequality. In addition to highlighting effects of parameters on the sample complexity of dictionary learning, we show the potential advantages of dictionary learning from tensor data and present unaddressed problems.

  • 8 - Computing Choice: Learning Distributions over Permutations

  • Edited by Miguel R. D. Rodrigues, University College London, Yonina C. Eldar, Weizmann Institute of Science, Israel
  • Book:
    Information-Theoretic Methods in Data Science
    Published online:
    22 March 2021
    Print publication:
    08 April 2021, pp 229-262

  • Summary

    We discuss the question of learning distributions over permutations of a given set of choices, options or items based on partial observations. This is central to capturing the so-called “choice’’ in a variety of contexts. The question of learning distributions over permutations arises beyond capturing “choice’’ too, e.g., tracking a collection of objects using noisy cameras, or aggregating ranking of web-pages using outcomes of multiple search engines. Here we focus on learning distributions over permutations from marginal distributions of two types: first-order marginals and pair-wise comparisons. We emphasize the ability to identify the entire distribution over permutations as well as the “best ranking’’.

Information-Theoretic Methods in Data Science
  • Edited by Miguel R. D. Rodrigues, Yonina C. Eldar
  • Published online:
    22 March 2021
    Print publication:
    08 April 2021
  • Learn about the state-of-the-art at the interface between information theory and data science with this first unified treatment of the subject. Written by leading experts in a clear, tutorial style, and using consistent notation and definitions throughout, it shows how information-theoretic methods are being used in data acquisition, data representation, data analysis, and statistics and machine learning. Coverage is broad, with chapters on signal acquisition, data compression, compressive sensing, data communication, representation learning, emerging topics in statistics, and much more. Each chapter includes a topic overview, definition of the key problems, emerging and open problems, and an extensive reference list, allowing readers to develop in-depth knowledge and understanding. Providing a thorough survey of the current research area and cutting-edge trends, this is essential reading for graduate students and researchers working in information theory, signal processing, machine learning, and statistics.

Page 52 of 117