Tianyuan Mathematical Center
in Southwest China

Gradient Tracking with Multiple Local SGD for Decentralized Non-Convex Optimization

报告题目：Gradient Tracking with Multiple Local SGD for Decentralized Non-Convex Optimization

报告专家：张纵辉（香港中文大学（深圳））

报告时间：2023年5月12日（星期五）10:00-11:00

报告地点：西南中心516

报告摘要：

The Gradient Tracking (GT) method, which solves decentralized optimization problems over a multi-agent network, is known to be robust against the inter-client variance caused by data heterogeneity. However, the GT method can be communication-intensive, requiring a large number of communication rounds of message exchange for convergence. To address this challenge, this work proposes a new communication-efficient GT algorithm called the Local Stochastic GT (LSGT) algorithm, which integrates the local stochastic gradient descent (local SGD) technique into the GT method. With LSGT, each agent can perform multiple SGD updates locally within each communication round. Theoretically, we establish the conditions under which our proposed LSGT algorithm enjoys the linear speedup brought by local SGD. Compared with the existing work, our analysis requires less restrictive conditions on the mixing matrix and algorithm stepsize. Moreover, it reveals that the local SGD not only reserves the resilience of the GT method against the data heterogeneity but also speeds up reducing the tracking error in the optimization process. The experimental results demonstrate that the proposed LSGT exhibits improved convergence speed and robust performance in various heterogeneous environments.

专家简介：

张纵辉，IEEE会士，国家高层次青年人才，现为香港中文大学（深圳）理工学院副教授、助理院长（主管教育）、广东省大数据计算基础理论与方法重点实验室副主任、深圳市大数据研究院研究员。目前与过去分别担任多个国际信号处理顶级期刊的资深编委和编委，IEEE信号处理协会（SPS）通信网络信号处理技术委员会委员、感知通信一体化工作组发起人与首届主席和IEEE SPS 董事会亚太区独立主席。

张教授专注于面向无线通信、机器学习的关键信号处理和优化方法的基础研究。“以优化及信号处理技术对无线通信的贡献”获得2015年IEEE通信学会亚太区杰出青年学者奖；在鲁棒波束赋形优化方面的基础性工作于2018年获得国际信号处理领域最具影响力的IEEE信号处理协会最佳论文奖；2021年以高效分布式优化方法的开创性工作第二次获得IEEE信号处理协会最佳论文奖;他也获得香港中文大学（深圳）理工学院首届卓越科研奖。近年来 主持和参与包括国家自然科学基金重点项目、面上项目、广东省重点项目、深圳市杰出青年项目以及华为、中兴等企业的横向项目10余项。其中“分布式基带架构的新型信道估计算法”获得华为2022年技术成果转化二等奖。

邀请人：宋恩彬