基於電腦視覺之實體課堂學生投入程度分析系統-以未來教室為例

Abstract

智慧教室是現代教育中快速發展的教學模式。隨著科技的迅速發展，數據化的應用同時為教育界帶來啟發，教學不再僅僅局限於傳統的課堂模式，而是逐漸融入了各種創新的教學方法和技術工具。在這一情境下，本研究旨在探索如何運用先進的電腦視覺技術，深入分析實體課堂中學生的投入程度，以此提升課堂的教學策略與學生的學習效果。透過在國立臺灣大學綜合教學館的未來教室中進行實體研究，我們不僅融合了教育和科技，還為教學評估提供了一個全新的角度。在這個背景下，本文介紹了我們的研究方法和主要成果。我們進一步描述了所構建的資料集以及設計的分析模型。藉由這些努力，我們致力於提供一個可行與實用的教學輔助系統，以幫助教育工作者更好地了解學生的學習狀況，並在教學過程中做出更明智的決策。 在本篇論文中，我們在未來教室中設置12隻4K解析度的攝影機與6隻指向型麥克風，收集合作課堂中各組學生的影音資料，其中共有三種不同類型，六堂不同時段的課程，其中參與人數約為24人，總計取得之資料約為67小時。為瞭解學生投入程度，我們與有教育相關背景的教授合作討論並制定學生投入程度判斷的規則，同時設計一套容易操作的影音資料標注系統，並招募30位標註者協助我們標註並建立我們的資料集。本研究建構了一套基於電腦視覺之實體課堂學生投入程度分析系統，以去識別化的方式擷取學生的人體骨架、聲音特徵以及環境物件，設計學生投入程度之分析模型。本研究強調學生投入程度的時間趨勢，交叉比對實際投入程度數據可達到0.76的相關係數，以此開發出一套能實際用於課堂中的教學輔助系統。我們同時也邀請三位老師進行訪談，他們皆認為這個系統很有幫助，根據數據化的呈現，除了課堂中能迅速掌握同學狀況外，課後也可以作為調整授課策略的參考，以達到最佳教學成效。

In the rapidly evolving landscape of modern education, the emergence of smart classrooms presents exciting opportunities. Technology has been revolutionizing education, expanding traditional boundaries, and enabling innovative teaching methods. In this study, we try to offer insights to enhance teaching practices by exploring the application of advanced computer vision technology to assess student engagement in physical classrooms. Conducted in the Future Classroom, Zonghe Lecture Building, National Taiwan University, this research combines education and technology to provide a fresh perspective on instructional assessment. Furthermore, we introduce the research methodology, key findings along with the constructed dataset, and our analytical model. We aim to develop a practical assistance system in order to assist instructors in understanding students’ learning conditions and also help them make better decisions during the teaching process. In this study, we deployed twelve 4K resolution cameras and six directional microphones to capture video and audio data from collaborative class sessions. This dataset, which enrolled approximately 24 participants, consists of 67 hours of audio-visual records. To assess students' engagement in the courses, we collaborated with professors who have educational backgrounds to establish engagement assessment rules and design an easy-to-use annotation system. We recruited 30 annotators to assist us in labeling and building our dataset This research introduces a novel computer vision-based system for analyzing student engagement in physical classrooms. Through anonymized data, which includes body skeletal information, audio features, and environmental objects, our model focuses on the trends in student engagement. It exhibits a significant correlation coefficient of 0.76 with ground truth data, indicating its potential as a practical assistant system for real-world classrooms. We also invited three teachers to have further discussions, and all of them found the system helpful. Moreover, they validate its utility, highlight its ability to quickly grasp student dynamics during class, and serve as a post-class reference for enhancing teaching strategies and optimizing instructional impact.