使用深度強化學習的5G網路服務多域資源編排和協作供應

Abstract

本文研究了延遲敏感和頻寬密集型5G服務的端到端多域編排。提供高質量的端到端服務需要來自多個域的無線電、計算和帶寬資源。然而，由於片面的資訊與不完整的資源控制，域之間的資源編排和交互成為基礎設施提供商的挑戰。此外，租戶需要滿足用戶的服務質量，且同時減少資源成本的消耗。在這項研究中，我們考慮了一種分散的場景，即維持基礎設施提供商的運營自治，並設計提供商之間的合作機制，藉由動態調整資源價格來增加收入。在端到端的資源編排框架下，我們開發智能和去中心化的解決方案。具體而言，對於租戶，我們提出了一種深度強化學習算法來優化資源編排問題。對於基礎設施提供商，我們提出了一種在線多智能體強化學習算法來優化其長期收入。實驗結果表明，我們的方法對於租戶在消耗成本和服務質量方面都有很好的表現。我們還表明，資源較少的基礎設施提供商可以通過合作機制獲得的收入比完整端到端資源的非合作提供商更多。

This study investigates end-to-end multi-domain orchestration for delay-sensitive and bandwidth-intensive 5G services. Providing high-quality end-to-end services require radio spectrum, link bandwidth, and computing resources from multiple domains. However, resource orchestration and interaction between domains are a challenge for infrastructure providers (InPs), who only have partial information and have incomplete control over resources. Moreover, tenants need to satisfy the quality of service (QoS) of their subscribers and simultaneously minimize resource costs. In this study, we consider a decentralized scenario that InP's operational autonomy is maintained and design a cooperation mechanism between InPs to increase their revenue by dynamically adjusting resource prices. We develop intelligent and decentralized solutions in the end-to-end multi-domain orchestration framework. Specifically, we propose a deep reinforcement learning algorithm for tenants to optimize the resource orchestration problem. For InPs, we propose an online multi-agent deep reinforcement learning algorithm to optimize their long-term revenue. The experimental results demonstrate that our method performs well in terms of both consumption costs and QoS for tenants. We also show that InPs with fewer resources can generate more revenue through our cooperative mechanism than non-cooperative InPs with complete end-to-end resources.