請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/19782
標題: | Hadoop 於異質平台之資源管理系統 Extending Resource Management System based on Heterogeneous Hadoop Yarn Platform |
作者: | Jian-Jang Shen 沈建志 |
指導教授: | 廖世偉 |
關鍵字: | 巨量資料,異質性分散式系統平台,圖形處理, Hadoop,MapReduce,Yarn,OpenCL,Aparapi, |
出版年 : | 2015 |
學位: | 碩士 |
摘要: | Apache Hadoop近年來的蓬勃發展廣泛用在巨量資料的應用上, 但 Hadoop 系統框架在CPU上所遇到的效能瓶頸時常為人所詬病, 如果要將 Hadoop 應用程 式移植到圖形處理器上達到效能提升的目的, 程序員必須要花很多額外的心力。 在這篇理論當中我們將利用Aparapi 程式庫來做到將 Hadoop 應用程式移植到圖 形處理器上執行, 並且探討 Hadoop YARN 框架在異質性平台上的資源管理。 Apache Hadoop allows developers for the distributed processing of large data sets across clusters of computers using simple programming models. The booming of Apache Hadoop solves many kinds of big data problems, and it is very suitable for parallel processing. But the poor performance of Hadoop applications due to the bottlenecks of computing is always reviled. Our research will proposed a framework which combines Haddop YARN and GPU, porting Aparapi libraries into YARN system for computing resources management in heterogeneous platforms. Extended the Application Master, which is a core component in YARN architecture to act as a role of resources request decision maker based on our scheduling algorithm. Besides, we adopt a preemptive, locality-aware task scheduling mechanism to fairly share CPU AND GPU resources. In the experiments, we show the overall speedup of an application, and analyze the effects to performance. |
URI: | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/19782 |
全文授權: | 未授權 |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-104-1.pdf 目前未授權公開取用 | 1.03 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。