混合式NoSQL資料庫系統之負載平衡

Abstract

在現今的NoSQL資料庫系統中,不同的系統會在一致性、可用性及分割容忍度間做優化的選擇。有的系統會有同時存取客戶端數量上的限制,其他的則可能要求記憶體必須跟存在硬碟中的資料一樣大。不同的設計策略可能會導致各種不同工作量設定下相異的反應延遲。為了兼得各設計策略之長處,有些公司便將混合式的資料庫應用在他們的程式上。在這樣的系統中應用程式的開發者必須預先定義策略來指派各表格該對應到哪個資料庫中。經常使用的表格可能會放在回應較快的節點上,其他的則留在可靠的系統上做長駐性存放。當表格中一些不常存取的資料突然來了超出預期的存取量,這樣的系統就會無法應付及搬移資料到回應較快的節點上。 在這篇論文,我們提出用負載平衡元件來動態偵測過熱的節點並搬移資料至有餘裕輸出的節點上的方法。我們還提供了一個可擴展/有彈性的資料庫界面以接上不同的資料庫,並期望維護者能直接進行創建、讀取、更新、刪除或是其他進階的操作而不需要再做策略上的調整。

In the field of NoSQL database systems nowadays, different systems have to make choices on whether to optimize for consistency, availability, and partition tolerance. Some of them has limitations on the number of concurrent served clients. Others might require the memory to be as large as the data in disk. Different design principles would lead to different latencies under different workloads and queries per second (QPS). To benefit from multiple design principles, some companies deploy hybrid databases for their applications. However, the application developers have to predefine schemes which assign tables to databases. Frequently used tables stay in more responsive nodes, while others stay in more reliable systems for permanent storage. When some parts of less-frequently accessed tables get unexpected amount of access accidently, this kind of systems cannot accomodate the changes and migrate data to more responsive nodes. In this thesis, we propose a load balancer with capabilites to dynamically detect hot spot node and migrate data to nodes with spare throughput capabilites. We also provide an extendable/flexible database interface to attach to different databases, and expect maintainers to directly do CRUD or other advanced operations without additional tuning on the schema.