We address the problem of learning an achievable lower bound on classification error from a labeled sample. We establish a framework for this meta-learning problem, which we call benchmark learning. Benchmark learning leads to an accurate data-driven predictor of performance of a Bayes optimal classifier without having to construct the classifier and without assuming any parametric model for the data. The resultant predictor can be used to establish whether it is possible to improve classification performance of a specific classifier. It also yields a stopping rule for sequentially trained classifiers. In addition, The talk will cover relevant background, theory, algorithms, and applications of benchmark learning.
Alfred O. Hero III received the B.S. (summa cum laude) from Boston University (1980) and the Ph.D from Princeton University (1984), both in Electrical Engineering. Since 1984 he has been with the University of Michigan, Ann Arbor, where he is the John H. Holland Distinguished University Professor of Electrical Engineering and Computer Science and the R. Jamison and Betty Williams Professor of Engineering. His primary appointment is in the Department of Electrical Engineering and Computer Science and he also has appointments, by courtesy, in the Department of Biomedical Engineering and the Department of Statistics. He is a Section Editor of the SIAM Journal on Mathematics of Data Science and a Senior Editor of the IEEE Journal on Selected Topics in Signal Processing. He is on the editorial board of the Harvard Data Science Review (HDSR). He serves as moderator for the Electrical Engineering and Systems Science category of the arXiv. He is co-General Chair of the IEEE International Symposium on Information Theory (ISIT).
He was founding Co-Director of the University’s Michigan Institute for Data Science (MIDAS) (2015-2018). From 2008-2013 he held the Digiteo Chaire d’Excellence at the Ecole Superieure d’Electricite, Gif-sur-Yvette, France. He is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) and the Society for Industrial and Applied Mathematics (SIAM). Several of his research articles have received best paper awards. Alfred Hero was awarded the University of Michigan Distinguished Faculty Achievement Award (2011), the Stephen S. Attwood Excellence in Engineering Award (2017), and the H. Scott Fogler Award for Professional Leadership and Service (2018). He received the IEEE Signal Processing Society Meritorious Service Award (1998), the IEEE Third Millennium Medal (2000), and the IEEE Signal Processing Society Technical Achievement Award (2014). He received the 2015 Society Award from the IEEE Signal Processing Society and he received the 2020 Fourier Award from the IEEE. Alfred Hero was President of the IEEE Signal Processing Society (2006-2008) and was on the Board of Directors of the IEEE (2009-2011) where he served as Director of Division IX (Signals and Applications). He served on the IEEE TAB Nominations and Appointments Committee (2012-2014). Alfred Hero is currently a member of the Big Data Special Interest Group (SIG) of the IEEE Signal Processing Society. Since 2011 he has been a member of the Committee on Applied and Theoretical Statistics (CATS) of the US National Academies of Science.
Alfred Hero’s recent research interests are in high dimensional spatio-temporal data, multi-modal data integration, statistical signal processing, and machine learning. Of particular interest are applications to social networks, network security and forensics, computer vision, and personalized health.