Abstract: A fundamental task in machine learning and data science is to learn low-dimensional representations from high-dimensional data. It has found wide applications in practice, such as MRI, face recognition, and community detection, to name a few. However, the optimization problems encountered in these applications are often non-convex. In this talk, we will discuss two different low-dimensional non-convex problems and propose efficient methods for solving them by exploring low-dimensional structures. The first one is to recover community structures in the stochastic block model. We propose a projected power method initialized by orthogonal iterations for solving this problem. We show that in the logarithmic sparsity regime of the problem, the proposed two-stage method can exactly recover the two communities down to the information-theoretic limit. The second one is to identify subspace structures in subspace clustering. In this project, we analyze the convergence and recovery performance of the K-subspaces method in the semi-random union of subspaces model. We show that if the initial assignment of the KSS method lies within a neighborhood of a true clustering, it converges at a superlinear rate and finds the correct clustering.
Speaker's bio:
Peng Wang is a postdoctoral research fellow in the Department of Electrical Engineering and Computer Science at University of Michigan from 2021. His research interests mainly lie in optimization theory and algorithms for non-convex optimization with low-dimensional structures and their applications in deep learning, machine learning, and data science. Currently, he is devoted to studying optimization landscape of deep neural networks. His research contributions have been recognized by top-tier journals Mathematical Programming, SIAM Journal on Optimization, and top machine learning conferences ICML, NeurIPS, and AISTATS. Before this, he is a postdoctoral research associate in the Department of Systems Engineering and Engineering Management at the Chinese University of Hong Kong. He received his Ph.D.'s degree at the Chinese University of Hong Kong in 2020 and his bachelor's degree at Beijing University of Posts and Telecommunications in 2016.