|本期目录/Table of Contents|

[1]康宇洋,刘为凯*.批量归一化的自适应联邦学习算法[J].武汉工程大学学报,2023,45(05):549-555.[doi:10.19843/j.cnki.CN42-1779/TQ.202304031]
　KANG Yuyang,LIU Weikai *.Adaptive Federated Learning Algorithm with Batch Normalization[J].Journal of Wuhan Institute of Technology,2023,45(05):549-555.[doi:10.19843/j.cnki.CN42-1779/TQ.202304031]
点击复制

批量归一化的自适应联邦学习算法(/HTML)
分享到：

《武汉工程大学学报》[ISSN:1674-2869/CN:42-1779/TQ]

卷:
45
期数:
2023年05期
页码:
549-555
栏目:
机电与信息工程
出版日期:
2023-11-17

文章信息/Info

Title:
Adaptive Federated Learning Algorithm with Batch Normalization
文章编号:
1674 - 2869（2023）05 - 0549 - 07
作者:
康宇洋; 刘为凯*
武汉工程大学光电信息与能源工程学院、数理学院，湖北 武汉 430205
Author(s):
KANG Yuyang; LIU Weikai *
School of Optical Information and Energy Engineering，School of Mathematic and Physics，
Wuhan Institute of Technology，Wuhan 430205， China
关键词:
联邦学习; 自适应参数; 批量归一化; 收敛速度
Keywords:
federated learning;  adaptive parameters;  batch normalization;  rate of convergence
分类号:
TN911
DOI:
10.19843/j.cnki.CN42-1779/TQ.202304031
文献标志码:
A
摘要:
针对联邦学习模型在训练过程中出现的客户端漂移以及协变量偏移的问题，提出一种基于批量归一化的自适应联邦学习算法。该算法融合参数自适应更新与批量归一化。在迭代的过程中，客户端本地模型通过自适应参数不断优化，从而缓解客户端漂移。通过批量归一化约束模型复杂度，算法收敛速度显著加快。使用时装数据集以及图像10分类数据集分别在卷积神经网络以及多层感知机网络模型上进行实验。结果表明，相较于经典的联邦平均算法，提出的算法在提升精度的同时加快了30%以上的收敛速度。在非独立同分布的数据实验中，该算法在设备低参与率的情况下也能够达到预期的效果。

Abstract:
Aiming at client drift and covariate shift during the training of federated learning， we proposed an adaptive federated learning method based on batch normalization. The method combines adaptive updating of parameters and batch normalization techniques. In the iteration process，client local models were gradually optimized by adaptively adjusting variable factors to mitigate client drift. The convergence rate was significantly enhanced by using batch normalization to constrain the complexity of models. With fashion-mixed national institute of standards and technology database and Canadian institute for advanced research-10 datasets，experiments were performed on both convolutional neural network and multi-layer perceptron network models. Compared with the classical federated average methods，experimental results demonstrate that the proposed method achieves higher accuracy and its convergence rate is enhanced by more than 30%. While it can also achieve promising?results with low participation rates of devices in non-identical-independent-distribution data experiments.

参考文献/References:

［1］ HARD A，RAO K， MATHEWS R， et al. Federated learning for mobile keyboard prediction［OL］.（2019-02-28）［2023-04-30］. https：//doi.org/10.48550/arXiv. 1811.03604.

［2］ LI T， SAHU A K， TALWALKAR A， et al. Federated learning：challenges，methods，and future directions［J］. IEEE Signal Processing Magazine，2020，37（3）：50-60.
［3］ BRISIMI T S， CHEN R D， MELA T， et al. Federated learning of predictive models from federated electronic health records［J］. International Journal of Medical Informatics，2018，112：59-67.
［4］ MCMAHAN?H B， MOORE ?E， RAMAGE?D， et al. Communication-efficient learning of deep networks from decentralized data［C］//SINGH A，ZHU X J. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics（AISTATS）. Cambridge：JMLR，2017：1273-1282.
［5］ YU H， YANG S， ZHU S H. Parallel restarted SGD with faster convergence and less communication：demystifying why model averaging works for deep learning［C］//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto，California：AAAI Press，2019：5693-5700.
［6］ ZENG M Y，WANG X M，PAN W J，et al. Heterogeneous training intensity for federated learning： a deep reinforcement learning approach［J］. IEEE Transactions on Network Science and Engineering，2023，10（2）：990-1002.
［7］ KARIMIREDDY S P，KALE S，MOHRI M，et al. Scaffold：stochastic controlled averaging for federated learning［OL］. （2021-04-09）［2023-04-30］. https：//doi.org/10.48550/arxiv.1910.06378.
［8］ SUN H R，HONG M Y. Distributed non-convex first-order optimization and information processing： lower complexity bounds and rate optimal algorithms［J］. IEEE Transactions on Signal Processing，2019，67（22）：5912-5928.
［9］ WANG S Q，TUOR T，SALONIDIS T，et al. Adaptive federated learning in resource constrained edge computing systems［J］. IEEE Journal on Selected Areas in Communications，2019，37（6）：1205-1221.
［10］ MA L S，SU W，LI X Z，et al. Heterogeneous data backup against early warning disasters in geo-distributed data center networks［J］. Journal of Optical Communications and Networking，2018，10（4）：376-385.
［11］ WANG Z G， ZHANG J W， CHANG T H， et al. Distributed stochastic consensus optimization with momentum for nonconvex nonsmooth problems［J］. IEEE Transactions on Signal Processing，2021，69：4486-4501.
［12］ MORAFAH M， VAHIDIAN S， WANG W J， et al. FLIS：clustered federated learning via inference similarity for non-IID data distribution［J］. IEEE Open Journal of the Computer Society，2023，4：109-120.
［13］ SHAO Y L，GüNDüZ D，LIEW S C. Federated edge learning with misaligned over-the-air computation［J］. IEEE Transactions on Wireless Communications，2022，21（6）：3951-3964.
［14］ LI X X，JIANG M R，ZHANG X F， et al. FedBN：federated learning on non-IID features via local batch normalization［C］// 9th International Conference on Learning Representations.［S.L.］：ICLR，2021.
［15］ FABOHUNGBE O， QIAN L. The effect of batch normalization on noise resistant property of deep learning models［J］. IEEE Access，2022，10：127728-127741.
［16］ CHEN Z D， DENG L， LI G Q，et al. Effective and efficient batch normalization using a few uncorrelated data for statistics estimation［J］. IEEE Transactions on Neural Networks and Learning Systems，2021，32（1）：348-362.
［17］ AWAIS M，IQBAL M T B，BAE S H，et al. Revisiting internal covariate shift for batch normalization［J］. IEEE Transactions on Neural Networks and Learning Systems，2021，32（11）：5082-5092.
［18］ XIAO H， RASUL K， VOLLGRAF R. Fashion-MNIST：a novel image dataset for benchmarking machine learning algorithms［OL］.（2017-09-15）［2023-04-30］.https：//doi.org/10.48550/arXiv.1708. 07747.
［19］ KRIZHEVSKY A. Learning multiple layers of features from tiny images［D］. Toronto：University of Toronto，2009.
［20］ 周治威，刘为凯，钟小颖. 自适应量化权重用于通信高效联邦学习［J］. 控制理论与应用，2022，39（10）：1961-1968.
［21］ 陈龙，张水平，王海晖，等. 基于多任务学习和知识图谱的面部表情识别［J］. 武汉工程大学学报，2021，43（6）：681-688.

相似文献/References:

备注/Memo

备注/Memo:
收稿日期：2023-04-27
基金项目：湖北省教育厅科学技术研究计划重点项目（D20131503）
作者简介：康宇洋，硕士研究生。E-mail：[email protected]
*通讯作者：刘为凯，博士，副教授。E-mail：[email protected]
引文格式：康宇洋，刘为凯. 批量归一化的自适应联邦学习算法［J］. 武汉工程大学学报，2023，45（5）：549-555.

更新日期/Last Update: 2023-10-25