Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Part Five Neural Networks
- Executive Summary
- 24 First Encounter with ReLU Networks
- 25 Expressiveness of Shallow Networks
- 26 Various Advantages of Depth
- 27 Tidbits on Neural Network Training
- Appendices
- References
- Index
27 - Tidbits on Neural Network Training
from Part Five - Neural Networks
Published online by Cambridge University Press: 21 April 2022
Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Part Five Neural Networks
- Executive Summary
- 24 First Encounter with ReLU Networks
- 25 Expressiveness of Shallow Networks
- 26 Various Advantages of Depth
- 27 Tidbits on Neural Network Training
- Appendices
- References
- Index
Summary
This chapter touches on some aspects related to the training of neural networks. First, a method called backpropagation is presented as a way to efficiently compute gradients in descent algorithms when deep networks are used. Next, the chapterconsiders shallow networks in the overparametrized regime, and it is proved that the empirical-risk landscape, despite its nonconvexity, features no strict local minimizers. Finally, convolutional neural networks are briefly mentioned.
Information
- Type
- Chapter
- Information
- Mathematical Pictures at a Data Science Exhibition , pp. 239 - 246Publisher: Cambridge University PressPrint publication year: 2022
Accessibility standard: Unknown
Why this information is here
This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.Accessibility Information
Accessibility compliance for the PDF of this book is currently unknown
and may be updated in the future.