Federated Semantic Communication for the Metaverse: Architecture, Evaluation, and Research Directions

Abstract

The Metaverse is a virtual digital space comprising multiple virtual subspaces that coexist and engage with the real world, establishing a collective virtual reality setting. A distinctive characteristic of the Metaverse is its inherent linkage between the physical and virtual domains; the former represents the physical space of the real world, while the latter comprises the virtual spaces within the Metaverse. This chapter investigates key technological aspects essential to the development of the Metaverse, including high-quality data processing, immersive experience mechanisms, large-scale access, and robust privacy and security. It presents a comprehensive analysis of the Metaverse’s service requirements, emphasising the necessity of low latency, large-scale data storage, robust security, and efficient data transmission to support immersive and reliable virtual environments. To address these requirements, a federated semantic communication framework is proposed, integrating semantic data transmission, semantic twins, and a Metaverse construction model trained via federated learning. The framework’s effectiveness is evaluated through simulations conducted on MNIST, KMNIST, and CIFAR10 datasets, using Peak Signal-to-Noise Ratio (PSNR) and classification accuracy as principal evaluation metrics. Experimental results demonstrate substantial improvements in transmission efficiency, data recovery quality, and intelligent recognition capabilities. Moreover, the framework achieves a low compression rate with minimal information distortion, thereby reducing transmission delays and enhancing the immersive quality of Metaverse environments. Finally, this chapter discusses future challenges and outlines potential research directions for advancing semantic communication, federated learning, and Metaverse technologies.