A Tour of Sparsity-Aware Learning -- Calling at: Online, Distributed, Robust And Dictionary Learning

Event Speaker
Sergios Theodoridis
Professor, Department of Informatics and Telecommunications, National and Kapodistrian University of Athens
Event Type
Colloquium
Date
Event Location
COVL 216
Event Description

Learning sparse models has been a topic at the forefront of research for the last ten years or so. Considerable effort has been invested in developing efficient schemes for the recovery of sparse signal/parameter vectors. Moreover, concepts that have originally been developed around the regression task have been extended to more general and difficult problems, such as low-rank matrix factorization for dimensionality reduction, robust learning in the presence of outliers, dictionary learning for “data-dependent” signal representation. Furthermore, online techniques for sparse modeling estimation are attracting an increasing interest, especially in the context of big data applications. Another area which is gaining in importance is distributed learning over graphs. An area, which was mainly inspired and born within the sensor network discipline, but now lends itself, nicely, for big data processing. In this talk, I touch upon all the previously mentioned problems. Sparse modeling of regression tasks is viewed in its online estimation facet, via convex analytic arguments, based on the set-theoretic framework; the emphasis is on very recent extensions of the theory to include non-convex related constraints, which impose sparsity on the model in a much more aggressive manner compared to the more standard, convex, l_1-norm related arguments. In spite of the involved non-convexity, still complexity per time iteration exhibits a linear dependence on the number of unknown parameters; furthermore, strong theoretical convergence results have been established. In the sequel, distributed learning techniques are reviewed with an emphasis on greedy-type batch as well as online versions. The task of robust learning in the presence of outliers is then reviewed and new methods, based on the explicit modeling of the outliers, in the context of sparsity-aware learning, will be presented. The new method, based on greedy-type arguments, enjoys a number of merits, compared to more classical techniques. Furthermore, strong theoretical results have been established, for the first time, in such a type of treatment of the robust estimation task. Finally, dictionary learning, in its very recent online and distributed processing framework, is discussed and new experimental as well as theoretical results will be presented

Speaker Biography

Sergios Theodoridis is currently Professor of Signal Processing and Machine Learning in the Department of Informatics and Telecommunications of the University of Athens. His research interests lie in the areas of Adaptive Algorithms, Distributed and Sparsity-Aware Learning, Machine Learning and Pattern Recognition, Signal Processing for Audio Processing and Retrieval. He is the author of the book “Machine Learning: A Bayesian and Optimization Perspective” Academic Press, 2015, the co-author of the best-selling book “Pattern Recognition”, Academic Press, 4th ed. 2009, the co-author of the book “Introduction to Pattern Recognition: A MATLAB Approach”, Academic Press, 2010, the co-editor of the book “Efficient Algorithms for Signal Processing and System Identification”, Prentice Hall 1993, and the co-author of three books in Greek, two of them for the Greek Open University. He currently serves as Editor-in-Chief for the IEEE Transactions on Signal Processing. He is Editor-in-Chief for the Signal Processing Book Series, Academic Press and co-Editor in Chief for the E-Reference Signal Processing, Elsevier. He is the co-author of seven papers that have received Best Paper Awards including the 2014 IEEE Signal Processing Magazine best paper award and the 2009 IEEE Computational Intelligence Society Transactions on Neural Networks Outstanding Paper Award. He is the recipient of the 2014 IEEE Signal Processing Society Education Award and the 2014 EURASIP Meritorious Service Award. He has served as an IEEE Signal Processing Society Distinguished Lecturer. He was Otto Monstead Guest Professor, Technical University of Denmark, 2012, and holder of the Excellence Chair, Dept. of Signal Processing and Communications, University Carlos III, Madrid, Spain, 2011. He serves (2014-2016) as Distinguished Lecturer for the IEEE Circuits and Systems Society. He has served as President of the European Association for Signal Processing (EURASIP), as a member of the Board of Governors for the IEEE CAS Society, as a member of the Board of Governors (Member-at-Large) of the IEEE SP Society and as a Chair of the Signal Processing Theory and Methods (SPTM) technical committee of IEEE SPS. He is Fellow of IET, a Corresponding Fellow of the Royal Society of Edinburgh (RSE), a Fellow of EURASIP and a Fellow of IEEE.