Edgar Dobriban
- Assistant Professor of Statistics
Contact Information
- Primary Email:
dobriban@wharton.upenn.edu
- office Address:
465 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104
Research Interests: Statistics and machine learning
Overview
The two main interests in my group are:
- the efficient statistical analysis of “big data” using advanced tools, such as those from random matrix theory
- theoretical analysis of modern machine learning, including deep learning
- data augmentation
- weight normalization
- stochastic gradient descent and flow
- sketching and random projections, [1], [2], [3], [4]
- distributed learning: [1], [2]
- adversarial robustness
- retraining of ML models
The group is always looking to expand. We are recruiting PhD students at Penn to work on problems in statistics and machine learning. PhD applicants interested to work with me should mention this on their application. Please apply through both the Statistics department and the AMCS program, as it gives higher chances for admission.
Seminar class in Fall 2019: Topics in Deep Learning (STAT-991), surveying advanced topics in deep learning research based on student presentations. See the Github page for the class materials.
Education (cv):
- PhD in Statistics, Stanford University, 2017. Advisor: David Donoho
- BA in Mathematics, Princeton University, 2012.
Recent news:
- Sept 2020: Co-organizing online workshop on Equivariance and Data Augmentation with Kostas Daniilidis, Sept 4. Join us! Part of activities for NSF Tripods center FinPenn. Youtube playlist.
- Aug 2020: Edgar is a Co-PI on NSF-Simons Mathematical and Scientific Foundations of Deep Learning award THEORINET (PI: Rene Vidal, JHU). Looking forward to many exciting collaborations that this will enable!
- Aug 2020: Talks at JSM 2020: See slides for talk on stochastic gradient flow, and slides prepared as a discussant for the invited session on the theory of deep learning.
- June 2020: SIAM Mathematics of Data Science minisymposium “High-Dimensional PCA in the High-Noise Regime” co-organized with William Leeb and Amit Singer. Link available from the SIAM webpage closer to the date. Please check the spreadsheet linked under “You can find the schedule of these talks here.”
- June 2020: New paper on provable tradeoffs in adversarially robust classification with Hamed Hassani, David Hong, Alexander Robey
- June 2020: Paper on distributed learning with Yue Sheng accepted at the Annals of Statistics.
- June 2020: Papers accepted at ICML 2020: “WONDER: Weighted one-shot distributed ridge regression” (short version), The Implicit Regularization of Stochastic Gradient Flow for Least Squares, and “DeltaGrad: Rapid retraining of machine learning models” (with Yinjun Wu and Susan Davidson).
- May 2020: Paper on How to reduce dimension with PCA and random projections? with Fan Yang, Sifan Liu, and David P. Woodruff.
- Apr 2020: Paper on “WONDER: Weighted one-shot distributed ridge regression” with Yue Sheng accepted to JMLR.
- Mar 2020: New paper on stochastic gradient descent and flow with Alnur Ali and Ryan Tibshirani.
- Feb 2020: New paper on iterative sketching with Jonathan Lacotte, Sifan Liu, and Mert Pilanci.
- Feb 2020: Talk at UMass Amherst (cancelled due to Covid-19).
- Jan 2020: Edgar becomes an affiliate of the Warren Center.
- Dec 2019: Edgar will be teaching Continuing Education course “Machine Learning and Deep Learning,” with Annie Qu and Xiao Wang, at the 2020 Joint Statistical Meetings (JSM). Also teaching short course “Deep Learning in Statistics” at SLDS 2020.
- Dec 2019: Ridge regression paper with Sifan Liu accepted as a spotlight presentation to ICLR 2020 (top 6% of papers).
- Dec 2019: New GPU machines arrive.
- Nov 2019: New paper on Implicit Regularization of Normalization Methods
- Oct 2019: Reading group/group meeting starts.
- Sept-Nov 2019: Edgar is visiting the IAS for the Special year on Statistics, Machine Learning, and Optimization.
- Sept 2019: PCA paper accepted to the Annals of Statistics
- Sept 2019: Multiple testing paper accepted to Biometrika
- Sept 2019: Sketching paper accepted to NeurIPS 2019
- Aug 2019: Edgar contributes to ASA statement “Comment on Statistics, as a Discipline and Practice, in AI Research and Development“
- Aug 2019: David Hong joins the group. Welcome David!
- July 2019: NSF TRIPODS awarded to Edgar as a co-PI, with Alejandro Ribeiro (PI, ESE), and co-PIs Robert Ghrist (Math), Kostas Daniilidis (CIS), Saswati Sarkar (ESE)
- July 2019: New paper on data augmentation with Shuxiao Chen and Jane Lee.
- Summer 2019: Undergraduate researchers Tianle Liu (THU), Shuo Xie (PKU), and Weichen Zheng (Penn) work in the group
- June-Aug 2019: Edgar is attending the Summer program on the Foundations of Deep Learning at the Simons Institute.
- March 2019: New paper on distributed learning with Yue Sheng.
Miscellanea:
- I use Twitter to keep up with new research.
- I grew up in Romania, and speak Hungarian as a first language (the real spelling of my name is Dobribán Edgár). These two countries are and were the origin of many great mathematicians and statisticians, including John von Neumann, Abraham Wald, Paul Erdos, Peter Bickel, Dan-Virgil Voiculescu, etc…
Research
Talk slides: GitHub. Google Scholar.
Yinjun Wu, Edgar Dobriban, Susan Davidson (2020), DeltaGrad: Rapid retraining of machine learning models, International Conference on Machine Learning (ICML) 2020.
Fan Yang, Sifan Liu, Edgar Dobriban, David P. Woodruff, How to reduce dimension with PCA and random projections?.
Sifan Liu and Edgar Dobriban (2020), Ridge regression: Structure, cross-validation, and sketching, International Conference on Learning Representations (ICLR).
Alnur Ali, Edgar Dobriban, Ryan J. Tibshirani (2020), The implicit regularization of stochastic gradient flow for least squares, International Conference on Machine Learning (ICML) 2020.
Jonathan Lacotte, Sifan Liu, Edgar Dobriban, Mert Pilanci (Working), Limiting spectrum of randomized hadamard transform and optimal iterative sketching methods.
Shuxiao Chen, Edgar Dobriban, Jane H Lee (Working), A group-theoretic framework for data augmentation.
Edgar Dobriban and Yue Sheng (2020), WONDER: Weighted one-shot distributed ridge regression in high dimensions, Journal of Machine Learning Research (JMLR), 52.
Edgar Dobriban (2020), Fast closed testing for exchangeable local tests, Biometrika, (to appear).
Description: To appear in Biometrika.
Edgar Dobriban, William Leeb, Amit Singer (2020), Optimal prediction in the linearly transformed spiked model, The Annals of Statistics, 48 (1), pp. 491-513.
Description: This paper supersedes the older Dobriban, Leeb, Singer manuscript "PCA from noisy, linearly reduced data: the diagonal case".
Edgar Dobriban (2020), Permutation methods for factor analysis and PCA, The Annals of Statistics, (to appear).
Teaching
Awards and Honors
- T.W. Anderson Theory of Statistics Dissertation Award, Department of Statistics, Stanford University, 2017
- Howard Hughes Medical Institute International Student Graduate Research Fellowship, 2015
- Stanford Department of Statistics Teaching Award, 2013
- Middleton Miller ’29 Prize for best independent work in mathematics, Princeton University, 2012
Miscellaneous
This page has links to methods from my papers. Feel free to contact me if you are interested to use them.
ePCA: github
The ePCA method for principal component analysis of exponential family data, e.g. Poisson-modeled count data. (with L.T. Liu);
EigenEdge: github
Methods for working with large random data matrices, including
- Computing eigenvalue distributions of covariance matrices (general Marchenko-Pastur distributions).
- Optimal statistics for testing in principal component analysis.
- Tools for spiked covariance models: spike and cosine descriptors, optimal shrinkers.
pweight : github R. github Matlab
P-value weighting techniques for multiple hypothesis testing. These can improve power in multiple testing, if there is prior information about the individual effect sizes. Includes the iGWAS method for Genome-Wide Association Studies.