請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/92193完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 簡韶逸 | zh_TW |
| dc.contributor.advisor | Shao-Yi Chien | en |
| dc.contributor.author | 洪愷縴 | zh_TW |
| dc.contributor.author | Kai-Chien Hung | en |
| dc.date.accessioned | 2024-03-08T16:14:08Z | - |
| dc.date.available | 2024-03-09 | - |
| dc.date.copyright | 2024-03-08 | - |
| dc.date.issued | 2024 | - |
| dc.date.submitted | 2024-02-05 | - |
| dc.identifier.citation | [1] Y. Cheng, H. Wang, Y. Bao, and F. Lu, “Appearance-based gaze estimation with deep learning: A review and benchmark,” arXiv preprint arXiv:2104.12668, 2021.
[2] L. R. Young and D. Sheena, “Survey of eye movement recording methods,” Behavior research methods & instrumentation, vol. 7, no. 5, pp. 397–429, 1975. [3] O. H. Mowrer, T. C. Ruch, and N. E. Miller, “The corneo-retinal potential difference as the basis of the galvanometric method of recording eye movements,” American Journal of Physiology-Legacy Content, vol. 114, no. 2, pp. 423–428, 1935. [4] E. Schott, “Uber die registrierung des nystagmus und anderer augenbewegungen verm itteles des saitengalvanometers,” Deut Arch Fur Klin Med, vol. 140, pp. 79–90, 1922. [5] D. W. Hansen and Q. Ji, “In the eye of the beholder: A survey of models for eyes and gaze,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 32, no. 3, pp. 478–500, 2009. [6] J. Chen and Q. Ji, “3D gaze estimation with a single camera without IR illumination,” in Proceedings of IEEE Conference on International Conference on Pattern Recognition (ICPR), 2008, pp. 1–4. [7] E. D. Guestrin and M. Eizenman, “General theory of remote gaze estimation using the pupil center and corneal reflections,” IEEE Transactions on Biomedical Engineering (TBE), vol. 53, no. 6, pp. 1124–1133, 2006. [8] Z. Zhu and Q. Ji, “Novel eye gaze tracking techniques under natural head movement,” IEEE Transactions on Biomedical Engineering (TBE), vol. 54, no. 12, pp. 2246–2260, 2007. [9] R. Valenti, N. Sebe, and T. Gevers, “Combining head pose and eye location information for gaze estimation,” IEEE Transactions on Image Processing (TIP), vol. 21, no. 2, pp. 802–815, 2011. [10] K. Alberto Funes Mora and J.-M. Odobez, “Geometric generative gaze estimation (g3e) for remote RGB-D cameras,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1773–1780. [11] C. Hennessey, B. Noureddin, and P. Lawrence, “A single-camera eye-gaze tracking system with free head motion,” in Proceedings of Symposium on Eye Tracking Research and Applications (ETRA), 2006, pp. 87–94. [12] D. H. Yoo and M. J. Chung, “A novel non-intrusive eye gaze estimation using cross-ratio under large head motion,” Computer Vision and Image Understanding (CVIU), vol. 98, no. 1, pp. 25–51, 2005. [13] Z. Zhu and Q. Ji, “Eye gaze tracking under natural head movements,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, 2005, pp. 918–923. [14] M. X. Huang, J. Li, G. Ngai, and H. V. Leong, “Screenglint: Practical, in-situ gaze estimation on smartphones,” in Proceedings of ACM CHI Conference on Human Factors in Computing Systems (CHI), 2017, pp. 2546–2557. [15] C. H. Morimoto and M. R. Mimica, “Eye gaze tracking techniques for interactive applications,” Computer Vision and Image Understanding (CVIU), vol. 98, no. 1, pp. 4–24, 2005. [16] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015. [17] P. S. Holzman, L. R. Proctor, D. L. Levy, N. J. Yasillo, H. Y. Meltzer, and S. W. Hurt, “Eye-tracking dysfunctions in schizophrenic patients and their relatives,” Archives of General Psychiatry (AGP), vol. 31, no. 2, pp. 143–151, 1974. [18] D. M. Stampe, “Heuristic filtering and reliable calibration methods for video-based pupil-tracking systems,” Behavior Research Methods, Instruments, & Computers (BRMIC), vol. 25, pp. 137–142, 1993. [19] Q. Ji and X. Yang, “Real-time eye, gaze, and face pose tracking for monitoring driver vigilance,” Real-Time Imaging, vol. 8, no. 5, pp. 357–377, 2002. [20] K.-H. Tan, D. J. Kriegman, and N. Ahuja, “Appearance-based eye gaze estimation,” in Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV), 2002, pp. 191–195. [21] F. Lu, Y. Sugano, T. Okabe, and Y. Sato, “Adaptive linear regression for appearance-based gaze estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 36, no. 10, pp. 2033–2046, 2014. [22] A. Colombo, A. Rognini, O. C. Santos, P. J. Clery, A. Van De Ville, and O. Blanke, “Perceived size and visual feedback modulate self-avatar’s embodiment,” Scientific Reports, vol. 7, no. 1, p. 1, 2017. [23] M. M. Biehl, R. R. Anderson, and J. F. Pitts, “The effect of visual feedback on performance of a closed-loop tracking task,” Journal of Motor Behavior, vol. 2, no. 1, pp. 3–12, 1970. [24] M. Tamura, M. Utsugi, H. Enzan, Y. Uchida, and S. Ishizawa, “Time constant of visually evoked cortical potentials to moving visual stimuli in rabbits,” Experimental Brain Research, vol. 17, no. 3, pp. 243–251, 1973. [25] J. T. Devlin and R. A. Poldrack, “In praise of tedious anatomy,” NeuroImage, vol. 37, no. 4, pp. 1033–1041, 2007. [26] H. P. Chan, K. T. Ng, and K. S. Tan, “The interocular transfer of motion aftereffect in cats,” Investigative Ophthalmology & Visual Science (IOVS), vol. 20, no. 2, pp. 217–220, 1981. [27] R. Steinman, M. Haddad, P. Skavenski, and R. Wyman, “Miniature eye movement,” Science, vol. 181, no. 4102, pp. 810–819, 1973. [28] R. L. Carpenter, R. P. G. Van Gisbergen, and A. C. H. Van Opstal, “The influence of eye position on the electrically elicited saccadic eye movement,” Experimental Brain Research, vol. 84, no. 1, pp. 221–225, 1991. [29] R. J. Van Wezel, G. M. Duijnhouwer, P. R. Van den Berg, and J. R. A. M. Van de Grind, “Relative binocular disparity as a depth cue,” The Journal of Neuroscience, vol. 18, no. 15, pp. 5978–5990, 1998. [30] R. J. Van Wezel and A. C. H. Van Opstal, “Reaching to visual targets: the physiological basis of visual feedback,” Vision Research, vol. 36, no. 21, pp. 2779–2797, 1996. [31] S. Park, S. D. Mello, P. Molchanov, U. Iqbal, O. Hilliges, and J. Kautz, “Few-shot adaptive gaze estimation,” in Proceedings of IEEE International Conference on Computer Vision (ICCV), 2019, pp. 9368–9377. [32] M. Xu, H. Wang, Y. Liu, and F. Lu, “Vulnerability of appearance-based gaze estimation,” arXiv preprint arXiv:2103.13134, 2021. [33] Y. Bao, Y. Cheng, Y. Liu, and F. Lu, “Adaptive feature fusion network for gaze tracking in mobile tablets,” in Proceedings of IEEE Conference on International Conference on Pattern Recognition (ICPR), 2021, pp. 9936–9943. [34] X. Zhang, Y. Sugano, M. Fritz, and A. Bulling, “Mpiigaze: Real-world dataset and deep appearance-based gaze estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 41, no. 1, pp. 162–175, 2017. [35] Y. Cheng and F. Lu, “Gaze estimation using transformer,” in Proceedings of IEEE Conference on International Conference on Pattern Recognition (ICPR), 2022, pp. 3341–3347. [36] X. Zhang, S. Park, T. Beeler, D. Bradley, S. Tang, and O. Hilliges, “Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation,” in Proceedings of European Conference on Computer Vision (ECCV), 2020, pp. 365–381. [37] T. Fischer, H. J. Chang, and Y. Demiris, “Rt-gene: Real-time eye gaze estimation in natural environments,” in Proceedings of European Conference on Computer Vision (ECCV), 2018, pp. 334–352. [38] B. A. Smith, Q. Yin, S. K. Feiner, and S. K. Nayar, “Gaze locking: passive eye contact detection for human-object interaction,” in Proceedings of ACM Symposium on User Interface Software and Technology (UIST), 2013, pp. 271–280. [39] Y. Sugano, Y. Matsushita, and Y. Sato, “Learning-by-synthesis for appearance-based 3d gaze estimation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1821–1828. [40] K. A. Funes Mora, F. Monay, and J.-M. Odobez, “Eyediap: A database for the development and evaluation of gaze estimation algorithms from rgb and rgb-d cameras,” in Proceedings of Symposium on Eye Tracking Research and Applications (ETRA), 2014, pp. 255–258. [41] Z. Chen and B. E. Shi, “Towards high performance low complexity calibration in appearance based gaze estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 1, pp. 1174–1188, 2022. [42] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in Proceedings of European Conference on Computer Vision (ECCV), 2018, pp. 3–19. [43] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014. [44] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. [45] C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. Yong, J. Lee et al., “Mediapipe: A framework for perceiving and processing reality,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. [46] K. Takahashi, S. Nobuhara, and T. Matsuyama, “Mirror-based camera pose estimation using an orthogonality constraint,” IPSJ Transactions on Computer Vision and Applications, vol. 8, pp. 11–19, 2016. [47] D. E. King, “Dlib-ml: A machine learning toolkit,” The Journal of Machine Learning Research, vol. 10, pp. 1755–1758, 2009. [48] V. Lepetit, F. Moreno-Noguer, and P. Fua, “Ep n p: An accurate o (n) solution to the p n p problem,” International Journal of Computer Vision, vol. 81, pp. 155–166, 2009. [49] G. Bradski, “The opencv library— dr. dobb’s,” Journal of Software Tools, 2010. [50] R. Rodrigues, J. P. Barreto, and U. Nunes, “Camera pose estimation using images of planar mirror reflections,” in Proceedings of European Conference on Computer Vision (ECCV), 2010, pp. 382–395. [51] J. He, K. Pham, N. Valliappan, P. Xu, C. Roberts, D. Lagun, and V. Navalkakkam, “On-device few-shot personalization for real-time gaze estimation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops(ICCVW), 2019, pp. 1149–1158. [52] T. Guo, Y. Liu, H. Zhang, X. Liu, Y. Kwak, B. In Yoo, J.-J. Han, and C. Choi, “A generalized and robust method towards practical gaze estimation on smart phone,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops(ICCVW), 2019, pp. 1131–1139. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/92193 | - |
| dc.description.abstract | 眼動追蹤技術是一項非常重要的工具，廣泛應用於科學研究及商業應用。其不僅能提供關於個體注意力、動機以及生理/心理狀態等訊息，更能在人機互動介面中提供更自然的方式，貼近個體需求。想像一下，若能透過眼睛便能操控電腦，取代傳統繁複的滑鼠操作，這種直覺且快速的使用者輸入方式，不僅將能為世界帶來顯著的改變，更能為更多族群提供友善與便利。
然而，市面上現有的眼動追蹤技術，多需額外的儀器或複雜的安裝及步驟設定，才能在個人電腦上使用。為了簡化這繁瑣的過程，此領域已有相關研究應運而生，希望能利用手機、平板及筆記型電腦內建的網路相機來捕捉使用者的臉部外觀，並直接預測其視線目標，進而實現即時控制滑鼠的便利性。 有鑑於眼動追蹤技術具有巨大的未來潛力及應用發展層面，我們架設了一套利用內建相機的即時眼動追蹤系統，提供最直接、最簡單的使用方式。另外，我們收集了能幫助我們設計之演算法訓練的資料集，並提出了一個加入注意力機制的輕量化模型，以達到在移動裝置上更具應用潛力的效能。在實驗中，我們的架構在三個著名的公開資料集中實現了最先進的性能，並在實驗各個指標上展現了我們提出的方法的實用性與可應用性。我們期望我們提出的即時眼動追蹤系統，能成為未來人機互動領域的重要里程碑。 | zh_TW |
| dc.description.abstract | Eye tracking is an important tool in various fields, including scientific research and commercial applications. However, existing eye tracking devices in the market often need extra equipment and complicated setup procedures to be used on personal computers. In an effort to make this process simpler, recent research in this field aims to use the built-in cameras of smartphones, tablets, and laptops to capture users'' face appearance. The goal is to directly predict where they are looking, making it easy to control the computer in real-time.
Recognizing the huge future potential and developmental possibilities of eye tracking technology, we have developed a practical real-time eye tracking system using built-in cameras, providing the most direct and simple user experience. Additionally, we collected our own dataset to help train algorithms designed by ourselves. We also propose a lightweight model that includes an attention mechanism to work better on portable devices. Our proposed method demonstrates practicality and applicability across various indicators in the experiments. We hope that our proposed eye tracking system will become an important step forward in how people interact with computers in the future. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2024-03-08T16:14:07Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2024-03-08T16:14:08Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Abstract (P.i)
List of Figures (P.v) List of Tables (P.vii) 1 Introduction (P.1) 1.1 Introduction of Eye Tracking (P.1) 1.2 Challenges (P.2) 1.3 Contribution (P.4) 1.4 Thesis Organization (P.6) 2 Related Work (P.7) 2.1 Evolution and Categorization of Gaze Estimation (P.7) 2.1.1 3D Eye Model Recovery Methods (P.8) 2.1.2 2D Eye Feature Regression Methods (P.9) 2.1.3 Appearance-based Methods (P.9) 2.2 Deep Learning for Appearance-based Gaze Estimation (P.10) 3 Proposed Method (P.15) 3.1 Rethinking the Eye Tracking System in the Whole Pipeline (P.15) 3.2 NTU-LaptopGaze Dataset (P.17) 3.2.1 Discussion of Existing Gaze Estimation Datasets (P.17) 3.2.2 Data Collection Procedure (P.19) 3.2.3 Dataset Characteristics (P.21) 3.3 Algorithm and Framework (P.25) 3.3.1 Unified Eye Tracking System On All Devices (P.26) 3.3.2 Camera-Anchored Lightweight Differential Method (P.26) 3.4 Eye Tracking System Pipeline (P.36) 3.4.1 User’s Interface and Pipeline in Eye Tracking System (P.36) 3.4.2 Replace Data Pre-processing Process with Coordinate Information (P.39) 3.4.3 Add Feature Cache to Make Model Prediction Faster (P.42) 4 Experimental Results (P.45) 4.1 Description of Datasets (P.45) 4.2 Implementation Details (P.46) 4.3 Comparisons With Existing Methods on Datasets (P.47) 4.3.1 Comparisons with Other Methods (P.47) 4.3.2 Comparisons with Other Methods on NTU-LaptopGaze (P.49) 4.4 Effectiveness of Jitter Loss (P.50) 4.5 Ablation Study of Proposed Method (P.53) 4.6 Discussion to Make System Faster (P.55) 5 Conclusion (P.57) Reference (P.59) | - |
| dc.language.iso | en | - |
| dc.subject | 基於外觀的凝視估計 | zh_TW |
| dc.subject | 深度學習 | zh_TW |
| dc.subject | 凝視預測 | zh_TW |
| dc.subject | 即時凝視估計 | zh_TW |
| dc.subject | 眼動追蹤系統 | zh_TW |
| dc.subject | 眼動追踪 | zh_TW |
| dc.subject | Eye Tracking | en |
| dc.subject | Gaze Prediction | en |
| dc.subject | Deep Learning | en |
| dc.subject | Appearance-based Gaze Estimation | en |
| dc.subject | Eye tracking System | en |
| dc.subject | Real-time Gaze Estimation | en |
| dc.title | 實境場景中基於外觀的遠端眼動追蹤全面架構： 系統實作與資料集整合 | zh_TW |
| dc.title | A Comprehensive Framework for Appearance-Based Remote Eye Tracking in the Wild: Practical System Implementation and Dataset Integration | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 112-1 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 塗偉志;曹昱;陳冠文 | zh_TW |
| dc.contributor.oralexamcommittee | Wei-Chih Tu;Yu Tsao;Kuan-Wen Chen | en |
| dc.subject.keyword | 基於外觀的凝視估計,眼動追踪,深度學習,凝視預測,即時凝視估計,眼動追蹤系統, | zh_TW |
| dc.subject.keyword | Appearance-based Gaze Estimation,Eye Tracking,Deep Learning,Gaze Prediction,Real-time Gaze Estimation,Eye tracking System, | en |
| dc.relation.page | 65 | - |
| dc.identifier.doi | 10.6342/NTU202400482 | - |
| dc.rights.note | 同意授權(限校園內公開) | - |
| dc.date.accepted | 2024-02-07 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 電子工程學研究所 | - |
| dc.date.embargo-lift | 2025-02-01 | - |
| 顯示於系所單位： | 電子工程學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-112-1.pdf 授權僅限NTU校內IP使用（校園外請利用VPN校外連線服務） | 13.01 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。