2 results
2 - Sparsity-aware distributed learning
- from Part I - Mathematical foundations
-
- By Symeon Chouvardas, University of Athens, Greece, Yannis Kopsinis, University of Athens, Greece, Sergios Theodoridis, University of Athens, Greece
- Edited by Shuguang Cui, Texas A & M University, Alfred O. Hero, III, University of Michigan, Ann Arbor, Zhi-Quan Luo, University of Minnesota, José M. F. Moura, Carnegie Mellon University, Pennsylvania
-
- Book:
- Big Data over Networks
- Published online:
- 18 December 2015
- Print publication:
- 14 January 2016, pp 37-65
-
- Chapter
- Export citation
-
Summary
In this chapter, the problem of sparsity-aware distributed learning is studied. In particular, we consider the setup of an ad-hoc network, the nodes of which are tasked to estimate, in a collaborative way, a sparse parameter vector of interest. Both batch and online algorithms will be discussed. In the batch learning context, the distributed LASSO algorithm and a distributed greedy technique will be presented. Furthermore, an LMS-based sparsity promoting algorithm, revolving around the l1 norm, as well as a greedy distributed LMS will be discussed. Moreover, a set-theoretic sparsity promoting distributed technique will be examined. Finally, the performance of the presented algorithms will be validated in several scenarios.
Introduction
The volume of data captured worldwide is growing at an exponential rate posing certain challenges regarding their processing and analysis. Data mining, regression, and prediction/forecasting have played a leading role in learning insights and extracting useful information from raw data. The employment of such techniques covers a wide range of applications in several areas such as biomedical, econometrics, forecasting sales models, content preference, etc. The massive amount of data produced together with their increased complexity (new types of data emerge) as well as their involvement in the Internet of Things [1] paradigm call for further advances in already established machine learning techniques in order to cope with the new challenges.
Even though data tend to live in high-dimensional spaces, they often exhibit a high degree of redundancy; that is, their useful information can be represented by using a number of attributes much lower compared to their original dimensionality. Often, this redundancy can be effectively exploited by treating the data in a transformed domain, in which they can be represented by sparse models; that is, models comprising a few nonzero parameters. Besides, sparsity is an attribute that is met in a plethora of models, modeling natural signals, since nature tends to be parsimonious. Such sparse structures can be effectively exploited in big data applications in order to reduce processing demands. The advent of compressed sensing led to novel theoretical as well as algorithmic tools, which can be efficiently employed for sparsity-aware learning, e.g. [2–7].
In many cases, processing of large amount of data is not only cumbersome but might be proved to be infeasible due to lack of processing power and/or of storage capabilities.
Literature review of spatio-temporal database models
- NIKOS PELEKIS, BABIS THEODOULIDIS, IOANNIS KOPANAKIS, YANNIS THEODORIDIS
-
- Journal:
- The Knowledge Engineering Review / Volume 19 / Issue 3 / September 2004
- Published online by Cambridge University Press:
- 17 June 2005, pp. 235-274
-
- Article
- Export citation
-
Recent efforts in spatial and temporal data models and database systems have attempted to achieve an appropriate kind of interaction between the two areas. This paper reviews the different types of spatio-temporal data models that have been proposed in the literature as well as new theories and concepts that have emerged. It provides an overview of previous achievements within the domain and critically evaluates the various approaches through the use of a case study and the construction of a comparison framework. This comparative review is followed by a comprehensive description of the new lines of research that emanate from the latest efforts inside the spatio-temporal research community.