A major challenge in statistical learning involves developing models that can effectively leverage the structure of a problem and generalize well. Achieving this goal critically depends on the precise selection and application of suitable regularization methods. Although there have been significant advancements in understanding explicit regularization techniques, such as Lasso and nuclear norm minimization, which have profoundly impacted the field in recent years, developing regularization approaches—especially those that are implicit or algorithmic—remains a difficult task.
In this talk, we will address this challenge by exploring the use of mirror descent. Employing tools from probability and optimization, we will introduce a structured framework for designing mirror maps and Bregman divergences. This framework enables mirror descent to attain optimal statistical rates in some settings in linear and kernel regression. If time permits, we will also briefly showcase the application of mirror descent in reinforcement learning, specifically focusing on the use of neural networks to learn mirror maps for policy optimization.