VOFiRM: 以RDMA和FUSE針對虛擬機設計的高彈性遠端記憶體

Abstract

記憶體分解解決了資料中心長期存在的不平衡的記憶體使用和較低的平均記憶體利用率等問題，其通過將部分記憶體從低利用率節點中解離，使具有較高記憶體需求的節點可以通過高速網絡遠端使用。另一方面，虛擬化技術也提倡資源共享，且其作為靈活的資源管理和快速佈署的手段，在雲端中得到廣泛應用。然而，雖然結合記憶體分解和虛擬化來實現更高效和靈活的資源使用似乎是可行的，但現有的解決方案通常需要修改作業系統核心，並且不允許應用程式參與遠端和本地記憶體的管理，這可能導致兼容性問題和/或低效率的記憶體共享。因此，我們提出了一個針對虛擬機設計的高彈性遠端記憶體 (VOFiRM) 的框架，該框架通過使用者空間檔案系統和遠端直接記憶體存取為虛擬機提供遠端記憶體，並支援按需求進行熱插拔的功能。由於該框架使用使用者空間檔案系統，它不需要修改虛擬機和虛擬機監視器，從而簡化了部署並減少了兼容性上的問題。遠端直接記憶體存取的使用至關重要，因為它可以加速遠端記憶體訪問並大大降低本地和遠端節點上的 CPU 的負載。 作為概念驗證，我們在 QEMU 上實現了 VOFiRM 的原型，並展示了系統管理員如何通過 QEMU 的監視器提供的接口附加/分離 pc-dimm 設備的檔案後端記憶體對象，使其能在不中斷虛擬機上的應用程式的情況下動態擴展/收縮虛擬機的遠程記憶體。我們從原型中收集實驗結果，以評估 VOFiRM 在各種工作負載下的性能影響並分析效能瓶頸。雖然遠端記憶體訪問的延遲和頻寬取決於網絡介面，但遠端記憶體分頁可以由作業系統核心緩存在本地，有效縮短訪問時間並提高讀取密集型應用程式的頻寬。此外，我們使用記憶體鍵值存儲系統，Redis，來進行應用案例研究，以了解 VOFiRM 在各種讀/寫比率下的表現，並演示 VOFiRM 的用戶如何調整記憶體配置以實現最佳性價比。

Memory disaggregation solves the long-existing memory imbalance and low average utilization problems in datacenters by decoupling parts of memory from low-utilization nodes so that memory-hungry nodes may use them remotely through a high-speed network. At the same time, virtualization technologies also advocate sharing of resources and are popularly used in the cloud as a means for flexible resource management and quickly service deployment. However, while combining memory disaggregation and virtualization to achieve more efficient and flexible resource usage seems viable, existing solutions often require modifications of operating system kernels and do not allow the applications to participate in the management of remote and local memories, which can lead to compatibility issues and/or inefficient memory sharing. Thus, we propose a framework called Virtual-Machine-Oriented Flexible Remote Memory (VOFiRM), which provides virtual machines with remote memories through Filesystem in USErspace (FUSE) and remote direct memory access (RDMA) with on-demand hotplug/hotunplug capabilities. As the proposed framework uses FUSE, it does not require kernel modifications in the virtual machine and hypervisor, which eases the deployment with less compatibility issues. The use of RDMA is critical as it accelerates remote memory accesses and greatly reduces the CPU overheads on local and remote nodes. As a proof of concept, we implement a prototype of VOFiRM on QEMU and show how a system administrator can dynamically extend/shrink the remote memories of a virtual machine without interrupting the applications on the virtual machine by attaching/detaching file-backend memory objects associated with pc-dimm devices via the interface provided by the QEMU monitor. We collect experimental results from the prototype to evaluate the performance impact of VOFiRM for various benchmark workloads and analyze the performance bottlenecks. While the latency and bandwidth for remote memoryaccess depend on the network interface, the remote memory pages can be cached locally by the kernel to effectively shorten the access time and improve the bandwidth for read-intensive applications. Furthermore, we conduct an application case study with Redis, an in-memory key-value store, to understand how VOFiRM performs under the various read/write ratios and demonstrate how the user of VOFiRM can adjust the memory configuration to achieve the best cost-performance.