Mixture of Linear Models Co-supervised by Deep Neural Networks

Abstract
Deep neural networks (DNNs) often achieve state-of-the-art prediction accuracy
across many applications. However, their adoption in certain domains is hindered by
the inherent complexity of DNN models, which poses significant challenges to their
interpretability. In contrast, linear models, such as logistic regression, are considered
highly interpretable but tend to have lower accuracy. Our goal is to develop mechanisms for balancing interpretability and accuracy to bridge the gap between
explainable linear models and black-box models. Specifically, we propose a new
approach termed Mixture of Linear Models (MLM) for regression or classification,
whose estimation is guided by a pre-trained DNN, acting as a proxy for the optimal
prediction function. Visualization methods and quantitative approaches have been
developed for interpretation. Experiments show that this new method can effectively
balance interpretability and accuracy. In some instances, MLM achieves comparable
accuracy to DNNs but significantly enhances interpretability. I will also briefly discuss
our more recent work on an EM-type algorithm to estimate MLM and its potential to
improve logistic regression for small datasets.

Speaker Bio
Jia Li, Department of Statistics, Penn State University. Jia Li is a Professor of Statistics and (by courtesy) Computer Science at the Pennsylvania State University. Her research interests include machine learning, artificial intelligence, probabilistic graph models, and image analysis. She was Editor-in-Chief for Statistical Analysis and Data Mining: the ASA Data Science Journal from January 2018 to December 2020. She worked as a Program Director at the National Science Foundation from 2011 to 2013, a Visiting Scientist at Google Labs in Pittsburgh from 2007 to 2008, a researcher at the Xerox Palo Alto Research Center from 1999 to 2000, and a Research Associate in the Computer Science Department at Stanford University in 1999. She received the M.Sc. degree in Electrical Engineering (1995), the M.Sc. degree in Statistics (1998), and the Ph.D. degree in Electrical Engineering (1999), from Stanford University. She is Fellow of IEEE and Fellow of ASA.