請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/4492完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 孫雅麗 | |
| dc.contributor.author | Zhen-Hou Zhou | en |
| dc.contributor.author | 周振澔 | zh_TW |
| dc.date.accessioned | 2021-05-14T17:42:40Z | - |
| dc.date.available | 2015-08-16 | |
| dc.date.available | 2021-05-14T17:42:40Z | - |
| dc.date.copyright | 2015-08-16 | |
| dc.date.issued | 2015 | |
| dc.date.submitted | 2015-08-15 | |
| dc.identifier.citation | 參考文獻
[1] Cisco, 'Cisco Visual Networking Index: Forecast and Methodology, 2013–2018,' 2014. [2] Y. S. S. Chang-Ming Wu, 'Visually Interactive Security Analysis of BigIP,' 2014. [3] Y. S. S. Wei-Ru Dai, 'Interactive Visualized Security Analysis System of Large Distributed Network Flow Data.,' 2014 [4] K. Shvachko, H. Kuang, S. Radia, and R. Chansler, 'The hadoop distributed file system,' in Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on, 2010, pp. 1-10. [5] D. J. Abadi, P. A. Boncz, and S. Harizopoulos, 'Column-oriented database systems,' Proceedings of the VLDB Endowment, vol. 2, pp. 1664-1665, 2009. [6] J. Dean and S. Ghemawat, 'MapReduce: simplified data processing on large clusters,' Communications of the ACM, vol. 51, pp. 107-113, 2008. [7] S. Ghemawat, H. Gobioff, and S.-T. Leung, 'The Google file system,' in ACM SIGOPS operating systems review, 2003, pp. 29-43. [8] A. Bialecki, M. Cafarella, D. Cutting, and O. O’MALLEY, 'Hadoop: a framework for running applications on large clusters built of commodity hardware,' Wiki at http://lucene. apache. org/hadoop, vol. 11, 2005. [9] G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, et al., 'Pregel: a system for large-scale graph processing,' in Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, 2010, pp. 135-146. [10] M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, 'Spark: cluster computing with working sets,' in Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, 2010, pp. 10-10. [11] M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, et al., 'Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing,' in Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012, pp. 2-2. [12] S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, et al., 'Dremel: interactive analysis of web-scale datasets,' Proceedings of the VLDB Endowment, vol. 3, pp. 330-339, 2010. [13] A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, et al., 'Hive: a warehousing solution over a map-reduce framework,' Proceedings of the VLDB Endowment, vol. 2, pp. 1626-1629, 2009. [14] C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, 'Pig latin: a not-so-foreign language for data processing,' in Proceedings of the 2008 ACM SIGMOD international conference on Management of data, 2008, pp. 1099-1110. [15] M. Kornacker and J. Erickson, 'Cloudera Impala: Real Time Queries in Apache Hadoop, For Real,' ht tp://blog. cloudera. com/blog/2012/10/cloudera-impala-real-time-queries-in-apache-hadoop-for-real, 2012. [16] A. Floratou, U. F. Minhas, and F. Ozcan, 'Sql-on-hadoop: Full circle back to shared-nothing database architectures,' Proceedings of the VLDB Endowment, vol. 7, 2014. [17] K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica, 'Sparrow: distributed, low latency scheduling,' in Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, 2013, pp. 69-84. [18] C. Gini, 'Measurement of inequality of incomes,' The Economic Journal, pp. 124-126, 1921. [19] S. Yitzhaki, 'Gini’s mean difference: A superior measure of variability for non-normal distributions,' Metron, vol. 61, pp. 285-316, 2003. [20] S. A. Cook, 'The complexity of theorem-proving procedures,' in Proceedings of the third annual ACM symposium on Theory of computing, 1971, pp. 151-158. [21] R. M. Karp, Reducibility among combinatorial problems: Springer, 1972. [22] M. E. McDowell, 'Multiprocessor Scheduling in the Presence of Communication Delay,' Master of Science Thesis. MIT, Dept. of Elec. Engineering and Comp. Science, Boston, 1989. [23] M. F. Tompkins, 'Optimization techniques for task allocation and scheduling in distributed multi-agent operations,' Massachusetts Institute of Technology, 2003. [24] M. Luo and H. Yokota, 'Comparing Hadoop and Fat-Btree based access method for small file I/O applications,' in Web-Age Information Management, ed: Springer, 2010, pp. 182-193. [25] H. Yokota, Y. Kanemasa, and J. Miyazaki, 'Fat-Btree: An update-conscious parallel directory structure,' in Data Engineering, 1999. Proceedings., 15th International Conference on, 1999, pp. 448-457. [26] B. Dong, Q. Zheng, F. Tian, K.-M. Chao, R. Ma, and R. Anane, 'An optimized approach for storing and accessing small files on cloud storage,' Journal of Network and Computer Applications, vol. 35, pp. 1847-1862, 2012. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/4492 | - |
| dc.description.abstract | 現今網路流量已以往無法想像的速度成長；網路犯罪亦隱身在龐大的網路流量中。為協助資安人員快速且有效率地為在網路流量中找出可做為呈堂供證的通聯記錄，我們提出了將網路流量視覺化的互動式查詢系統－NetActy。在本論文中對NetActy的互動性以及視覺化過程進行改進，藉由考慮節點間工作量的平衡以及Data Locailty，目的為了使計算節點執行時間平衡以達到互動程度的回應時間。本論文將工作量分配制定成一個Linear Programming問題，並提出經驗解－Algorithm 1以期在多項式時間內解決；視覺化部分，我們為每個查詢視圖做快取以及利用Multicast技術來加速處理。最後於實驗中，我們衡量Algorithm 1的效能確認其能夠在不違背Data Locality的情況下平衡節點間工作量；此外在視覺化部分所遇到的問題我們亦參考現行作業系統的做法來解決。 | zh_TW |
| dc.description.abstract | As the network volume grows rapidly, network crimes can hide behind the huge network traffic. In order to let IT security people find evidences fastly and effectively from such a huge network traffic, we proposed a interactive, visualable network query system－NetActy. In this thesis, we improve the interactivity and visualization process, by takeing the balance between workload and data locality into consider. We formulate the job assignment problem into a Linear Programming problem and solve it by a heuristic solution－Algorithm 1. In the last, we evaluate the performance of Algorithm 1 and make sure that Algorithm 1 can actually balance the workload without violating data locality. Besides, we solve the problem encountered in visualization part by applying current OS’s solution. | en |
| dc.description.provenance | Made available in DSpace on 2021-05-14T17:42:40Z (GMT). No. of bitstreams: 1 ntu-104-R02725012-1.pdf: 2762639 bytes, checksum: 6595085723b95c2a989af483f1db7d37 (MD5) Previous issue date: 2015 | en |
| dc.description.tableofcontents | 目錄
第一章 介紹 1 第一節 研究背景 1 第二節 研究問題 1 第三節 研究貢獻 2 3.1 NetActy儲存系統 3 3.2 NetActy系統執行架構 5 3.3 NetActy的效能問題 7 第二章 文獻探討 10 第一節 MapReduce 10 A. 設計目的： 10 B. 設計目標： 10 C. 設計細節： 11 第二節 Pregel 13 A. 設計目的： 13 B. 設計目標： 13 C. 與本論文差異： 14 第三節 Spark 15 A. 設計目的： 15 B. 設計目標： 15 C. 與本論文差異： 16 第四節 Dremel 17 A. 設計目的： 17 B. 設計目標： 17 C. 與本論文差異： 18 第五節 Impala 19 A. 設計目的： 19 B. 設計目標： 19 C. 與本論文差異： 21 第六節 Sparrow 21 A. 設計目的： 21 B. 設計目標： 22 C. 與本論文差異： 24 第三章 最小化工作完成時間差異之工作分配演算法 25 第一節 目標 25 第二節 問題定義 25 第三節 最小化計算節點執行時間之間差距的工作分配演算法 28 3.1 複雜度分析 28 3.2 Heuristic Solution 30 3.4 實驗設計 40 A. 實驗環境介紹 40 B. Query 40 C. 結果討論 43 3.5 檔案資料特性 44 3.6 計算節點之實體結構 48 3.7 Hierarchical Path改進 57 3.8 資料執行時間過久處理方式 57 第四章 搜尋結果視覺化之運算 66 第一節 方法一 66 第二節 視覺化介面的資料瀏覽：BRT Traversal 68 2.1 通訊模式 68 2.2 加速瀏覽回應時間 70 2.3 記憶體消耗量 71 第五章 結論與建議 74 | |
| dc.language.iso | zh-TW | |
| dc.subject | 流量視覺化 | zh_TW |
| dc.subject | 資安犯罪偵查 | zh_TW |
| dc.subject | 互動式查詢 | zh_TW |
| dc.subject | 大數據 | zh_TW |
| dc.subject | 工作分配 | zh_TW |
| dc.subject | 資料在地化 | zh_TW |
| dc.subject | Network security forensics | en |
| dc.subject | NetFlow records visualization | en |
| dc.subject | Data locality | en |
| dc.subject | Job assignment | en |
| dc.subject | Big data | en |
| dc.subject | Interactive query system | en |
| dc.title | 巨量網路資料之互動式安全分析系統 | zh_TW |
| dc.title | An Interactive Security Analysis System of Large Scale NetFlow Data | en |
| dc.type | Thesis | |
| dc.date.schoolyear | 103-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 陳孟彰,潘育群,洪士灝 | |
| dc.subject.keyword | 資安犯罪偵查,互動式查詢,大數據,工作分配,資料在地化,流量視覺化, | zh_TW |
| dc.subject.keyword | Network security forensics,Interactive query system,Big data,Job assignment,Data locality,NetFlow records visualization, | en |
| dc.relation.page | 77 | |
| dc.rights.note | 同意授權(全球公開) | |
| dc.date.accepted | 2015-08-15 | |
| dc.contributor.author-college | 管理學院 | zh_TW |
| dc.contributor.author-dept | 資訊管理學研究所 | zh_TW |
| 顯示於系所單位： | 資訊管理學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-104-1.pdf | 2.7 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。