تقرير
(Rectified Version) Push-LSVRG-UP: Distributed Stochastic Optimization over Unbalanced Directed Networks with Uncoordinated Triggered Probabilities
| العنوان: | (Rectified Version) Push-LSVRG-UP: Distributed Stochastic Optimization over Unbalanced Directed Networks with Uncoordinated Triggered Probabilities |
|---|---|
| المؤلفون: | Hu, Jinhui, Chen, Guo, Li, Huaqing, Shen, Zixiang, Zhang, Weidong |
| المصدر: | IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 10, NO. 2, 2023, PP. 934-950 |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science Mathematics |
| مصطلحات موضوعية: | Mathematics - Optimization and Control, Electrical Engineering and Systems Science - Systems and Control |
| الوصف: | Distributed stochastic optimization, arising in the crossing and integration of traditional stochastic optimization, distributed computing and storage, and network science, has advantages of high efficiency and a low per-iteration computational complexity in resolving large-scale optimization problems. This paper concentrates on resolving a large-scale convex finite-sum optimization problem in a multi-agent system over unbalanced directed networks. To tackle this problem in an efficient way, a distributed consensus optimization algorithm, adopting the push-sum technique and a distributed loopless stochastic variance-reduced gradient (LSVRG) method with uncoordinated triggered probabilities, is developed and named Push-LSVRG-UP. Each agent under this algorithmic framework performs only local computation and communicates only with its neighbors without leaking their private information. The convergence analysis of Push-LSVRG-UP is relied on analyzing the contraction relationships between four error terms associated with the multi-agent system. Theoretical results provide an explicit feasible range of the constant step-size, a linear convergence rate, and an iteration complexity of Push-LSVRG-UP when achieving the globally optimal solution. It is shown that Push-LSVRG-UP achieves the superior characteristics of accelerated linear convergence, fewer storage costs, and a lower per-iteration computational complexity than most existing works. Meanwhile, the introduction of an uncoordinated probabilistic triggered mechanism allows Push-LSVRG-UP to facilitate the independence and flexibility of agents in computing local batch gradients. In simulations, the practicability and improved performance of Push-LSVRG-UP are manifested via resolving two distributed learning problems based on real-world datasets. Comment: 16 pages, 30 figures |
| نوع الوثيقة: | Working Paper |
| DOI: | 10.1109/TNSE.2022.3225229 |
| الوصول الحر: | http://arxiv.org/abs/2305.09181Test |
| رقم الانضمام: | edsarx.2305.09181 |
| قاعدة البيانات: | arXiv |
| DOI: | 10.1109/TNSE.2022.3225229 |
|---|