於邊緣進行注重頻寬的推論:以實驗性方法分析影片超分析用於被壓縮影片之效果

Abstract

隨著深度學習的蓬勃發展，結合了強大深度神經網絡的行動裝置應用程式，特別是在擴增實境、混合實境和虛擬實境等等領域，正在蓬勃發展。然而，這些模型的計算需求往往超出了普通行動裝置的能力範圍，需要將神經網路運算移動至邊緣伺服器上。然而，這種方法會進一步衍生出延遲和網絡限制，這對邊緣運算輔助之移動擴增實境（Mobile Augmented Reality）中的實時性能尤其影響，特別是當上傳頻寬對於多數使用者來說是比較稀少的網路資源。

為了減輕頻寬的限制，本研究提出透過傳輸低畫質影片到邊緣伺服器後，再藉由超解析度（Super-Resolution）技術來增強影片品質與影像辨識（Object Detection）效能的方法。基於許多有關圖像質量對影像辨識的影響和超解析度在影片增強中之有效性的先前研究，我們探索了不同超解析度方法和影片壓縮率對使用邊緣運算做輔助之影像辨識的影響。我們的貢獻包括展現不同影片品質和超解析度水平上對於影片位元率與影像辨識性能之間的權衡，以及評估超解析度方法在低畫質影片上的效率。

With the proliferation of deep learning, mobile applications integrating robust deep neural networks, particularly in Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR), are on the rise. However, the computational demands of these models often exceed the capabilities of commodity mobile devices, necessitating offloading to edge servers. Yet, this approach introduces latency and network challenges, crucial for real-time performance in edge-assisted Mobile Augmented Reality (MAR), especially when uplink bandwidth is a scarce resource for most users.

To mitigate bandwidth constraints, this study proposes transmitting low-quality (LQ) videos, augmented with super-resolution (SR) techniques to enhance object detection. Drawing upon prior research on image quality's impact on object detection and the efficacy of SR in video enhancement, we explore various SR methods and video compression rates' effects on object detection in a MAR scenario. Our contributions include delineating the trade-offs between video bitrate and object detection performance across different quality and resolution levels and assessing the effectiveness of SR methods on LQ videos.