請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/88182
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 傅楸善 | zh_TW |
dc.contributor.advisor | Chiou-Shann Fuh | en |
dc.contributor.author | 林聖祐 | zh_TW |
dc.contributor.author | Sheng-Yu Lin | en |
dc.date.accessioned | 2023-08-08T16:40:17Z | - |
dc.date.available | 2023-11-09 | - |
dc.date.copyright | 2023-08-08 | - |
dc.date.issued | 2023 | - |
dc.date.submitted | 2023-06-29 | - |
dc.identifier.citation | B. F. Momin, “Current Status and Future Research Directions in Monitoring Vigilance of Individual or Mass Audience in Monotonous Working Environment,” 2012.
C. Berka, “EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks,” 2007. H. R. Schiffman, “Sensation and Perception. An Integrated Approach,” New York: John Wiley and Sons, Inc., 2001. H.-S. Shin, S.-J. Jung, J.-J. Kim, and W.-Y. Chung, “Real time car driver’s condition monitoring system,” 2010. I. Nasri, M. Karrouchi, H. Snoussi, K. Kassmi, and A. Messaoudi, “DistractNet: a deep convolutional neural network architecture for distracted driver classification,” 2022. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” CVPR, 2016. K. Bo-Chen, “Driver Monitoring System with Embedded Artificial Intelligence,” 2022. L.-C. Shi and B.-L. Lu, “Dynamic clustering for vigilance analysis based on EEG,” 2008. L. Pauly, D. Sankar, “Detection of Drowsiness based on HOG features and SVM classifiers”, 2015. L. Chen, Y. Zhao, J. Zhang, and J. Zou, “Automatic detection of alertness/drowsiness from physiological signals using wavelet-based nonlinear features and machine learning,” 2015. M. Akin, M. B. Kurt, N. Sezgin, and M. Bayram, “Estimating vigilance level by using EEG and EMG signals,” 2008. M. Shahverdy, M. Fathy, R. Berangi, and M. Sabokrou, “Driver behavior detection and classification using deep convolutional neural networks,” 2020. M. Ingre, T. Åkerstedt, B. Peters, A. Anund, and G. Kecklund, “Subjective sleepiness, simulated driving performance and blink duration: examining individual differences”, 2006. S. Masood, A. Rai, A. Aggarwal, M. N. Doja, and M. Ahmad, “Detecting distraction of drivers using Convolutional Neural Network”, 2020. S. Mohanty, S. V. Hegde, S. Prasad, and J. Manikandan, “Design of Real-time Drowsiness Detection System using Dlib”, 2019. S. Otmani, T. Pebayle, J. Roge, and A. Muzet, “Effect of driving duration and partial sleep deprivation on subsequent alertness and performance of car drivers”, 2005. T. Ouyang and H.-T. Lu, “Vigilance Analysis Based on Continuous Wavelet Transform of EEG Signals”, 2010. B. S. Jahromi, “Levels of Automation for Autonomous Ground Vehicles,”https://medium.com/@BabakShah/levels-of-automation-for-self-driving-cars-d410a4f679b7, 2023. Bulat, Adrian and Tzimiropoulos, Georgios, “How far are we from solving the 2D \& 3D Face Alignment problem?”, https://github.com/1adrianb/face-alignment, 2017. China National Standard (中华人民共和国国家标准), “Human Dimensions of Chinese Adults,” GB/T 10000-1988, 1988. Datik Irizar Group, “An industry review on driver fatigue systems for fleet operators”, https://www.datik.es/en/datik-3/news/an-industry-review-on-driver-fatigue-systems-for-fleet-operators-2, 2018. F. Lambert, “Tesla Reveals How It Will Use Camera inside Model 3 to Personalize In-Car Experience,”https://electrek.co/2019/07/24/tesla-use-camera-inside-cars-personalize-in-car-experience/, 2019. G. Jocher, “YOLOv5 by Ultralytics (Version 7.0),” https://doi.org/10.5281/zenodo.3908559, 2020. J. D. Ortega, N, Kose, P. Cañas, M.-A. Chao, A. Unnervik, M. Nieto, O. Otaegui, L. Salgado, “DMD: A Large-Scale Multi-modal Driver Monitoring Dataset for Attention and Alertness Analysis”, https://dmd.vicomtech.org/#activities, 2020. Kneron AI Documentations, https://doc.kneron.com/docs/#, 2019. Regulation (EU) 2019/2144 of the European parliament and of the council, “Commission Delegated Regulation (EU),” https://members.wto.org/crnattachments/2021/TBT/EEC/21_0621_00_e.pdf, 2021. S. Abtahi, M. Omidyeganeh, S. Shirmohammadi, B. Hariri, “YAWDD: YAWning Detection Dataset,” https://ieee-dataport.org/open-access/yawdd-yawning-detection-dataset, 2020. T. Ǻkerstedt, “Karolinska Sleepiness Scale,” https://www.med.upenn.edu/cbti/assets/user-content/documents/Karolinska%20Sleepiness%20Scale%20(KSS)%20Chapter.pdf, 1990. Wikipedia, “Driver Monitoring System,” https://en.wikipedia.org/wiki/Driver_monitoring_system, 2023. | - |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/88182 | - |
dc.description.abstract | 駕駛監測系統(DMS, Driver Monitoring System)可以通過即時影像,檢測駕駛頭部、面部和眼瞼運動的變化,來提示駕駛員他們的困倦或分心。中國及歐盟已規定該系統需配備於新出廠的車輛。
本論文提出一個基於YOLOv5s(You Only Look Once version 5 small)所設計的駕駛監測系統,並利用Kneron KL-520神經網路加速晶片運行於嵌入式裝置,能以低影像解析度,檢測駕駛閉眼、頭部姿態異常、打哈欠、使用手機、抽菸等行為。 Kneron KL-520是耐能(Kneron)在2019年推出的一款AI (Artificial Intelligence)晶片,約40 nm晶圓製程,算力為350 TOPS (Tera Operations Per Second),平均功耗降為300-500 mW,並且提供動態模型執行 (DME, Dynamic Model Execution),但對機器學習模型的大小及結構有嚴格限制。 為配合嵌入式系統的限制,本文將原本的YOLOv5s移除掉頭部及頸部等部分結構,在Ubuntu 18.0環境上透過自身收集的5萬張影像訓練,最後將模型佈署在配有Kneron KL-520開發板上,能將結果以平均5 FPS (Frames Per Second)顯示於顯示裝置上,即時對疲勞駕駛提出警告,各個種類的準確度(Accuracy)則可以達到90%以上。 | zh_TW |
dc.description.abstract | The Driver Monitoring System (DMS, Driver Monitoring System) can detect changes in the driver's head, face and eyelid movements through real-time images to alert drivers of their drowsiness or distraction. China and the European Union have stipulated that brand-new vehicles must be equipped with DMS system.
We propose a driver monitoring system based on YOLOv5s (You Only Look Once version 5 small) and use the Kneron KL-520 neural network accelerator chip to run on an embedded device, which can detect driving eyes closed, abnormal head posture, yawning, using mobile phones, smoking, and other behaviors. Kneron KL-520 is an AI (Artificial Intelligence) chip launched by Kneron in 2019, with 40 nm wafer process technology, a computing power of 350 TOPS, an average power consumption of 300-500 mW, and dynamic model execution. However, there are strict restrictions on the size and structure of machine learning model. To cope with the limitations of the embedded system, we remove some structures such as the head and neck from the original YOLOv5s, and train the model on the 50,000 images collected by ourselves in the Ubuntu 18.0 environment, and finally deploy machine learning model on the development board equipped with KL-520. The prediction result can be displayed on the display device with an average of 5 FPS (Frames Per Second), which can immediately warn fatigue drivers, and the accuracy of each category can reach more than 90%. | en |
dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2023-08-08T16:40:17Z No. of bitstreams: 0 | en |
dc.description.provenance | Made available in DSpace on 2023-08-08T16:40:17Z (GMT). No. of bitstreams: 0 | en |
dc.description.tableofcontents | 口試委員審定書 i
致謝 ii 中文摘要 iii ABSTRACT iv CONTENTS vi LIST OF FIGURES viii LIST OF TABLES xi Chapter 1 Introduction 1 1.1 Driver Monitoring System 1 1.2 Types of DMS 4 1.3 Challenges to Computer Vision-Based DMS 6 1.4 Present National Regulation 7 1.5 Thesis Organization 10 Chapter 2 Related Works 11 2.1 Physiological Signals 11 2.2 Facial Features 12 2.3 Driver Behaviors 12 Chapter 3 Background 14 3.1 You Only Look Once 14 3.2 You Only Look Once Version 5 15 3.3 Neural-network Processing Unit 16 3.4 Kneron AI 17 3.5 Kneron KL520 20 Chapter 4 Methodology 25 4.1 Workflow 25 4.2 Experiment Setup 26 4.3 Face Alignment Model & The Way We Labeling 28 4.4 Our DMS Model 32 4.5 Fatigue and Distraction Detection 36 4.6 Model Conversion 40 Chapter 5 Experiment Results 43 5.1 Dataset 43 5.2 Result 47 Chapter 6 Future Works 52 References 53 | - |
dc.language.iso | en | - |
dc.title | 林監督:駕駛監督系統用嵌入式系統 | zh_TW |
dc.title | LinDMS: Driver Monitoring System with Embedded System | en |
dc.type | Thesis | - |
dc.date.schoolyear | 111-2 | - |
dc.description.degree | 碩士 | - |
dc.contributor.oralexamcommittee | 邱立誠;方瓊瑤 | zh_TW |
dc.contributor.oralexamcommittee | Li-Cheng Chiu;Chiung-Yao Fang | en |
dc.subject.keyword | 駕駛者監控系統,嵌入系統,邊緣運算,人工智慧,影像處理, | zh_TW |
dc.subject.keyword | Driver Monitoring System,Embedded System,Edge Computing,Artificial Intelligence,Image Processing, | en |
dc.relation.page | 56 | - |
dc.identifier.doi | 10.6342/NTU202301045 | - |
dc.rights.note | 未授權 | - |
dc.date.accepted | 2023-06-30 | - |
dc.contributor.author-college | 電機資訊學院 | - |
dc.contributor.author-dept | 生醫電子與資訊學研究所 | - |
顯示於系所單位: | 生醫電子與資訊學研究所 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-111-2.pdf 目前未授權公開取用 | 3.46 MB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。