丁手勢：基於 3D 毫米波手勢定位的牆面互動系統

Abstract

隨著人機互動技術的迅速發展，智慧空間與互動系統的應用日益廣泛。牆面互動系統因其直觀的界面與大規模顯示特性，已在教育及公共空間等領域獲得廣泛應用。近年來，毫米波（mmWave）雷達技術在互動牆系統中的應用，成為克服傳統人機互動方法局限性的有力解決方案。本研究提出丁手勢 (TingGesture)，一個創新的系統，運用毫米波雷達技術實現互動牆應用中的精確手勢定位。該系統包含四個關鍵模組：點雲映射、熱圖去噪、階層式聚類演算法 (HDBSCAN)以及深度學習回歸校準，這些模組協同運作以實現高精度的手部位置追蹤。 本研究首先將毫米波點雲數據映射到熱圖上，接著採用雙軌中值模糊去雜訊策略，有效保留空間細節同時消除雜訊。隨後應用 HDBSCAN 聚類算法識別手部位置並移除殘餘雜訊，其中最佳參數通過窮舉搜索確定。為解決座標投影中的扇形失真和擴張問題，系統實施深度學習回歸模型進行校準。 實驗結果顯示 TingGesture 相較於傳統方法具有優越性能。在訓練路徑上，系統將均方根誤差（RMSE）從 14.376 公分顯著降低至 8.819 公分，在測試路徑上則從 13.153 公分降低至 8.632 公分。與雷達內建的人數計數算法相比，TingGesture在所有空間維度上都展現出更高的精確度，即使在各種傳輸延遲條件下，定位誤差增加仍能保持在 5%以下。 本研究展示了 TingGesture 在互動牆應用中實現精確手勢定位的有效性。通過全面的實驗驗證，該系統相較於傳統方法在定位精確度和運算效能方面都有顯著提升。

With the rapid advancement of human-computer interaction technologies, smart spaces and interactive systems have become increasingly prevalent in modern applications. Interactive wall systems, featuring intuitive interfaces and large-scale displays, have gained widespread adoption in educational, exhibition, and public spaces. In recent years, the integration of millimeter-wave (mmWave) radar technology into interactive wall systems has emerged as a promising solution to overcome the limitations of traditional human-computer interaction methods. This study introduces TingGesture, an innovative system leveraging mmWave radar for precise hand gesture localization in interactive wall applications. The methodology encompasses four key modules: Point Cloud Mapping, Heat Map Denoising, HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) Clustering, and Deep Learning Regression Calibration, working in concert to achieve high-precision hand position tracking. TingGesture first maps mmWave point cloud data onto a heat map, followed by a dual-track median blur denoising strategy that effectively preserves spatial details while eliminating noise. HDBSCAN clustering is then applied to identify hand positions and remove residual noise, with optimized parameters determined through exhaustive search. To address the fan-shaped distortion and expansion issues in coordinate projection, a deep learning regression model is implemented for calibration. Experimental results demonstrate TingGesture's superior performance compared with conventional methods. TingGesture achieves a significant reduction in Root Mean Square Error (RMSE) from 14.376 cm to 8.819 cm on the training path, and from 13.153 cm to 8.632 cm on the testing path. When compared with the radar's built-in people counting algorithm, TingGesture shows enhanced accuracy spatially, maintaining positioning error increases below 5% even under various transmission delay conditions. This research demonstrates the effectiveness of TingGesture in achieving precise hand gesture localization for interactive wall applications. Through comprehensive experimental validation, the system shows significant improvements in positioning accuracy and computational efficiency compared with conventional methods.