Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications

Abstract: The environment observations and learning experiences shared by the cooperative learning agents accelerate multi-agent reinforcement learning (MARL) with partial observations for mobile applications but the performance degrades due to the redundant and outdated observations under severe channel fading in wireless networks. In this paper, we propose an efficient communication scheme in MARL for mobile applications that enables each learning agent to optimize the cooperative agents and the learning parameters to integrate the shared information. The cooperative agents are chosen according to the learning environment observations, the channel states, and the task similarity with neighboring agents. The learning parameters are chosen based on the attention mechanism that exploits the correlation with the local observation to enhance the agent receptive field for efficient policy exploration. Neural networks with weights updated based on the learning factors determined by the task similarity are designed to further improve the learning efficiency. The performance bounds including the information gain from the learning agent cooperation, the communication cost and the utility are provided based on the Nash equilibrium of the cooperative MARL communication game. The proposed scheme is implemented in the anti-jamming video transmission of the unmanned aerial vehicle swarms to optimize the transmit channel and power and experimental results verify the performance gain over the benchmark.

Download paper here

[More information] Recommended citation: Zefang Lv, Liang Xiao, Yousong Du, Yunjun Zhu, Shuai Han, Yong-Jin Liu*.Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications,” in IEEE Transactions on Wireless Communications, doi: 10.1109/TWC.2024.3392608.