Generative principal component analysis and fast diffusion sampling
报告专家：刘昭强教授 （电子科技大学）
报告时间：2024年3月26日下午9:30-10:15
报告地点：国家天元数学西南中心516报告厅
报告摘要：In this talk, I will first briefly discuss the problem of principal component analysis with generative modeling assumptions. The key assumption is that the underlying signal lies near the range of a Lipschitz continuous generative model with bounded low-dimensional inputs. We propose a quadratic estimator, and show that it enjoys the optimal statistical rate. Moreover, we provide a variant of the classic power method, which projects the calculated data onto the range of the generative model during each iteration. We show that under suitable conditions, this method converges exponentially fast to a point achieving the above-mentioned statistical rate. We perform experiments on various image datasets to illustrate performance gains of our method to the classic power method and the truncated power method devised for sparse principal component analysis.
Next, I will briefly discuss our recent work on accelerating diffusion sampling with optimized time steps. Diffusion (generative) models (DMs) have shown remarkable performance in high-resolution image synthesis, but their sampling efficiency is still to be desired due to the typically large number of sampling steps. Recent advancements in high-order numerical ODE solvers for DMs have enabled the generation of high-quality images with much fewer sampling steps, but most sampling methods simply employ uniform time steps. To address this issue, we propose a general framework for designing an optimization problem that seeks more appropriate time steps for a specific numerical ODE solver for DMs. This optimization problem aims to minimize the distance between the ground-truth solution to the ODE and an approximate solution corresponding to the numerical solver. It can be efficiently solved using the constrained trust region method, taking less than 15 seconds. Our extensive experiments on both unconditional and conditional sampling using pixel- and latent-space DMs validate the effectiveness of the optimized time steps.
专家简介：刘昭强，电子科技大学计算机科学与工程学院教授、博士生导师，国家级青年人才。2013年本科毕业于清华大学数学系，2017年博士毕业于新加坡国立大学数学系。曾任新加坡国立大学电子电脑工程系和计算机系博士后研究员，以及香港华为诺亚方舟实验室人工智能研究员。研究方向为理论机器学习，目前主要从事扩散模型、大语言模型、高维逆问题等相关的理论分析。
邀请人：连增 唐庆粦
