請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/94666完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 莊裕澤 | zh_TW |
| dc.contributor.advisor | Yuh-Jzer Joung | en |
| dc.contributor.author | 王佑豪 | zh_TW |
| dc.contributor.author | Yu-Hao Wang | en |
| dc.date.accessioned | 2024-08-16T17:24:52Z | - |
| dc.date.available | 2024-08-17 | - |
| dc.date.copyright | 2024-08-16 | - |
| dc.date.issued | 2024 | - |
| dc.date.submitted | 2024-08-08 | - |
| dc.identifier.citation | [1] AWS. 什麼是日誌檔案. https://aws.amazon.com/tw/what-is/log-files/, 2023. Accessed: 2023-12-05.
[2] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov. Enriching word vectors with subword information, 2017. [3] R. Chalapathy and S. Chawla. Deep learning for anomaly detection: A survey. ArXiv, abs/1901.03407, 2019. [4] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR, 13–18 Jul 2020. [5] Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. V. Le, and R. Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. In Annual Meeting of the Association for Computational Linguistics, 2019. [6] L. Decker, D. Leite, F. Viola, and D. Bonacorsi. Comparison of evolving granular classifiers applied to anomaly detection for predictive maintenance in computing centers. In 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), pages 1–8, 2020. [7] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. [8] M. Du and F. Li. Spell: Streaming parsing of system event logs. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 859–864, 2016. [9] M. Du, F. Li, G. Zheng, and V. Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, page 1285– 1298, New York, NY, USA, 2017. Association for Computing Machinery. [10] K. Ethayarajh. How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In K. Inui, J. Jiang, V. Ng, and X. Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 55–65, Hong Kong, China, Nov. 2019. Association for Computational Linguistics. [11] Q. Fu, J.-G. Lou, Y. Wang, and J. Li. Execution anomaly detection in distributed systems through unstructured log analysis. In 2009 Ninth IEEE International Conference on Data Mining, pages 149–158, 2009. [12] T. Gao, X. Yao, and D. Chen. SimCSE: Simple contrastive learning of sentence embeddings. In M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic, Nov. 2021. Association for Computational Linguistics. [13] H. Guo, Y. Guo, J. Yang, J. Liu, Z. Li, T. Zheng, L. Zheng, W. Hou, and B. Zhang. Loglg: Weakly supervised log anomaly detection via log-event graph construction. In Database Systems for Advanced Applications: 28th International Conference, DASFAA 2023, Tianjin, China, April 17– 20, 2023, Proceedings, Part IV, page 490–501, Berlin, Heidelberg, 2023. Springer-Verlag. [14] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9726–9735, 2020. [15] P. He, J. Zhu, Z. Zheng, and M. R. Lyu. Drain: An online log parsing approach with fixed depth tree. In 2017 IEEE International Conference on Web Services (ICWS), pages 33–40, 2017. [16] S. He, J. Zhu, P. He, and M. R. Lyu. Experience report: System log analysis for anomaly detection. In 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pages 207–218, 2016. [17] A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon. A survey on contrastive self-supervised learning, 2021. [18] M. Jiang, C. Hou, A. Zheng, X. Hu, S. Han, H. Huang, X. He, P. S. Yu, and Y. Zhao. Weakly supervised anomaly detection: A survey, 2023. [19] T. Kudo and J. Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Conference on Empirical Methods in Natural Language Processing, 2018. [20] M. Landauer, S. Onder, F. Skopik, and M. Wurzenberger. Deep learning for anomaly detection in log data: A survey. Machine Learning with Applications, 12:100470, June 2023. [21] M. Landauer, F. Skopik, M. Wurzenberger, and A. Rauber. System log clustering approaches for cyber security applications: A survey. Comput. Secur., 92:101739, 2020. [22] M. Landauer, M. Wurzenberger, F. Skopik, G. Settanni, and P. Filzmoser. Dynamic log file analysis: An unsupervised cluster evolution approach for anomaly detection. Computers and Security, 79:94–116, 2018. [23] V. Le and H. Zhang. Log-based anomaly detection without log parsing. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 492–504, Los Alamitos, CA, USA, nov 2021. IEEE Computer Society. [24] V. Le and H. Zhang. Log-based anomaly detection with deep learning: How far are we? In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pages 1356–1367, Los Alamitos, CA, USA, may 2022. IEEE Computer Society. [25] B. Li, H. Zhou, J. He, M. Wang, Y. Yang, and L. Li. On the sentence embeddings from bert for semantic textual similarity. ArXiv, abs/2011.05864, 2020. [26] H.-J. Liao, C.-H. Richard Lin, Y.-C. Lin, and K.-Y. Tung. Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications, 36(1):16–24, 2013. [27] Q. Lin, H. Zhang, J.-G. Lou, Y. Zhang, and X. Chen. Log clustering based problem identification for online service systems. 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pages 102–111, 2016. [28] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019. [29] J.-G. Lou, Q. Fu, S. Yang, Y. Xu, and J. Li. Mining invariants from console logs for system problem detection. In Proceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, page 24, USA, 2010. USENIX Association. [30] W. Meng, Y. Liu, Y. Zhu, S. Zhang, D. Pei, Y. Liu, Y. Chen, R. Zhang, S. Tao, P. Sun, and R. Zhou. Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 4739–4745. International Joint Conferences on Artificial Intelligence Organization, 7 2019. [31] H. Mi, H. Wang, Y. Zhou, M. R.-T. Lyu, and H. Cai. Toward fine-grained, unsupervised, scalable performance diagnosis for production cloud computing systems. IEEE Transactions on Parallel and Distributed Systems, 24(6):1245–1255, 2013. [32] T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space, 2013. [33] M. Nagappan, K. Wu, and M. A. Vouk. Efficiently extracting operational profiles from execution logs using suffix arrays. In 2009 20th International Symposium on Software Reliability Engineering, pages 41–50, 2009. [34] S. Nedelkoski, J. Bogatinovski, A. Acker, J. Cardoso, and O. Kao. Self-supervised log parsing, 2020. [35] K. A. Nguyen, S. Schulte im Walde, and N. T. Vu. Integrating distributional lexical contrast into word embeddings for antonym-synonym distinction. In K. Erk and N. A. Smith, editors, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 454–459, Berlin, Germany, Aug. 2016. Association for Computational Linguistics. [36] A. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07), pages 575–584, 2007. [37] A. J. Oliner, A. Ganapathi, and W. Xu. Advances and challenges in log analysis. Communications of the ACM, 55:55 – 61, 2011. [38] J. Pennington, R. Socher, and C. Manning. GloVe: Global vectors for word representation. In A. Moschitti, B. Pang, and W. Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, Oct. 2014. Association for Computational Linguistics. [39] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. Deep contextualized word representations. In M. Walker, H. Ji, and A. Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. [40] J. E. Prewett. Analyzing cluster log files using logsurfer. 2003. [41] J. Qi, S. Huang, Z. Luan, C. J. Fung, H. Yang, and D. Qian. Loggpt: Exploring chatgpt for log-based anomaly detection. 2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), pages 273–280, 2023. [42] R. Sennrich, B. Haddow, and A. Birch. Neural machine translation of rare words with subword units. In K. Erk and N. A. Smith, editors, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany, Aug. 2016. Association for Computational Linguistics. [43] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15:1929–1958, 2014. [44] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. [45] T. Wang and P. Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, 2020. [46] Z. Wu, Y. Xiong, S. X. Yu, and D. Lin. Unsupervised feature learning via non-parametric instance discrimination. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3733–3742, 2018. [47] B. Xia, J. Yin, J. Xu, and Y. Li. Loggan: A sequence-based generative adversarial network for anomaly detection based on system logs. In International Conference on Science of Cyber Security, 2019. [48] B. Xu, N. Wang, T. Chen, and M. Li. Empirical evaluation of rectified activations in convolutional network, 2015. [49] W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP ’09, page 117– 132, New York, NY, USA, 2009. Association for Computing Machinery. [50] R. B. Yadav, P. S. Kumar, and S. V. Dhavale. A survey on log anomaly detection using deep learning. In 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), pages 1215–1220, 2020. [51] C. Zhang, X. Peng, C. Sha, K. Zhang, Z. Fu, X. Wu, Q. Lin, and D. Zhang. Deeptralog: Trace-log combined microservice anomaly detection through graph-based deep learning. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pages 623–634, 2022. [52] X. Zhang, Y. Xu, Q. Lin, B. Qiao, H. Zhang, Y. Dang, C. Xie, X. Yang, Q. Cheng, Z. Li, J. Chen, X. He, R. Yao, J.-G. Lou, M. Chintalapati, F. Shen, and D. Zhang. Robust log-based anomaly detection on unstable log data. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019, page 807– 817, New York, NY, USA, 2019. Association for Computing Machinery. [53] X. Zhou, X. Peng, T. Xie, J. Sun, C. Ji, W. Li, and D. Ding. Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study. IEEE Transactions on Software Engineering, 47:243–260, 2018. [54] J. Zhu, S. He, P. He, J. Liu, and M. R. Lyu. Loghub: A large collection of system log datasets for ai-driven log analytics. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE), pages 355–366, Los Alamitos, CA, USA, oct 2023. IEEE Computer Society. [55] J. Zhu, S. He, J. Liu, P. He, Q. Xie, Z. Zheng, and M. R. Lyu. Tools and benchmarks for automated log parsing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 121–130, Los Alamitos, CA, USA, may 2019. IEEE Computer Society. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/94666 | - |
| dc.description.abstract | 本研究針對系統日誌異常檢測問題提出了一個創新的二階段訓練框架，先利用日誌的模板（Template）資訊訓練一個基於 Contrastive Learning 的 Transformer 模型，將日誌直接轉換成一個特徵向量的資料點，我們會保證資料點能夠保留事件模板的資訊，並套用在下游的正常日誌之建模任務上，使得我們可以對異常日誌進行檢測。根據本論文的架構設計，我們可以在非監督式的異常檢測任務上獲得平均 94.84 的 F1 Score，比先前的 State-of-the-Art 模型 LogBERT 的表現（89.93）還要更高。文獻中的非監督式學習模型會將日誌轉換為模板後，使用一個 Embedding 層來將模板對應到特徵向量，而本研究會在推論時會直接使用日誌內容轉換為特徵向量，我們會在展示與其他模型的表現比較後，接著討論這個架構設計的合理性。其次，我們將提出一個方法，在推論時使用簡單的嵌入層替換掉 Transformer 模型，這樣的取代能夠大幅增加推論速度，使該論文架構能夠成為線上日誌異常檢測的優秀選擇。 | zh_TW |
| dc.description.abstract | This study proposes an innovative two-stage training framework for the problem of system log anomaly detection. First, we use the template information of the logs to train a Transformer embedder based on Contrastive Learning, directly converting the logs into feature vector data points. We ensure that the data points retain the event template information and apply them to the downstream task of modeling normal logs by training a simple LSTM network, allowing us to detect anomalous logs. According to the framework design in this paper, we achieve an average F1 Score of 94.84 in unsupervised anomaly detection tasks, which is higher than the previous state-of-the-art model LogBERT's performance (89.93). We will discuss our intuition and rationale in depth in the experiment part. Moreover, we propose a method to get rid of the embedder at inference time, making it a competitive option for online log anomaly detection system. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2024-08-16T17:24:51Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2024-08-16T17:24:52Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Acknowledgements i
Abstract ii 中文摘要 iii Contents iv List of Figures vi List of Tables viii Chapter 1 Introduction 1 1.1 Background and Motivation 1 1.2 Objective and Main Contribution 4 1.3 Thesis Organization 5 Chapter 2 Literature Review 6 2.1 Definitions of Logs and Log Parsing 6 2.2 Log Analysis 9 2.3 Towards Feature Representation Synthesis 12 2.3.1 Problems in Masked LM and Their Solutions 14 2.4 Anomaly Detection Module 17 2.4.1 Log Grouping 18 2.4.2 How Deep Learning Models Catch Anomaly 19 2.4.3 Comparison Between Different Models 21 Chapter 3 Methodology 23 3.1 Proposed Framework 23 3.1.1 Log Preprocessing 25 3.1.2 Log Parsing 26 3.1.3 Feature Representation Synthesizing 27 3.1.4 Log Grouping 33 3.1.5 Log Anomaly Detection 33 Chapter 4 Experiment Result 34 4.1 Research Datasets 35 4.2 Evaluation Metrics 36 4.3 Hyperparameters 37 4.4 Evaluation Results 40 4.5 Discussing the Validness of the Model 42 4.6 Remove Parameters To Prove Validness 48 4.6.1 Reduce Inference Time via Discarding Embedder 49 Chapter 5 Conclusion 52 References 55 Appendix A - Loss Function Formulation 64 Appendix B - Tokenizer Max Length Decision 65 | - |
| dc.language.iso | en | - |
| dc.title | 基於對比學習實現之自監督式系統日誌異常檢測 | zh_TW |
| dc.title | Framework for Self-Supervised Log Anomaly Detection Based on Contrastive Learning | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 112-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 楊立偉;陳建錦;陳以錚 | zh_TW |
| dc.contributor.oralexamcommittee | Li-wei Yang;Chien-Chin Chen;Yi-Cheng Chen | en |
| dc.subject.keyword | 系統日誌,異常檢測,非監督式學習,對比學習,時間序列分析, | zh_TW |
| dc.subject.keyword | System Log,Anomaly Detection,Unsupervised Learning,Contrastive Learning,Time Series Analysis, | en |
| dc.relation.page | 66 | - |
| dc.identifier.doi | 10.6342/NTU202402929 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2024-08-10 | - |
| dc.contributor.author-college | 管理學院 | - |
| dc.contributor.author-dept | 資訊管理學系 | - |
| dc.date.embargo-lift | 2029-07-31 | - |
| 顯示於系所單位： | 資訊管理學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-112-2.pdf 此日期後於網路公開 2029-07-31 | 2.08 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。