Widhi Yahya, Yuan-Cheng Lai, Ying-Dar Lin, Mahdin Rohmatillah, Binayak Kar
In a distributed edge system, such as the Central Office Re-architected as a Datacenter (CORD), traffic offloading is crucial for minimizing latency by redistributing traffic from overloaded edge sites to less congested sites. The control plane must make rapid decisions to accommodate fluctuating traffic rates. Traditional optimization methods, such as Simulated Annealing (SA), often require significant time to converge in this complex environment. This study proposes the Weighted Action Twin Delayed Deep Deterministic Policy Gradient (WA-TD3), a weighted action approach that optimizes the exploration stage to reduce the exploration space of the original TD3 algorithm in continuous action spaces, resulting in faster convergence. WA-TD3 enhances decision-making in edge offloading by effectively addressing the continuous search space problem and enabling quicker decision-making compared to traditional optimization methods. Evaluation results indicate that WA-TD3 achieves decision times measured in milliseconds, making it five orders of magnitude faster than SA, which has decision times ranging from minutes to hours, despite WA-TD3 exhibiting slightly higher average packet latency. In addition to its performance advantages, the offloading agent can be implemented in a centralized cloud (single agent) or across multiple core networks (multi-agent) with varying coverage. In CORD with 50 sites, the convergence times for SA, single-agent WA-TD3, and multi-agent WA-TD3 are 47,813 s, 1,302 s, and 929 s, respectively. © 2025 Elsevier Ltd
Faculty of Computer Science, Universitas Brawijaya, Malang, Indonesia; Department of Information Management, National Taiwan University of Science and Technology, Taipei, Taiwan; Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan; Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan; Department of electrical engineering, Universitas Brawijaya, Malang, Indonesia
