林融合：毫米波雷達與彩色相機融合用於人物追踪

Abstract

本論文提出林融合:一個用於室內人員追蹤的毫米波雷達與攝影機融合系統。毫米波雷達負責感知行走的人，可以同時對多個目標進行測距、測速以及方位角測量，從而得出人在空間中的具體位置與速度。同時，攝影機端利用深度學習中的物件偵測和追蹤的方法，獲得人在影像中的位置與唯一標籤。將這兩個裝置得到的資訊進行融合，可以得到在空間中每個人的唯一識別符號和具體位置。 為了驗證系統的有效性，我們設計了一系列實驗。在室內環境中收集了毫米波雷達和攝影機的資料，並使用我們的融合系統對這些資料進行處理。我們分別對坐標校正方法和位置估計方法進行了實驗，以確保融合精度。我們的系統性能評估指標是觀察每個人的唯一識別符號是否發生變化。 本研究提出了一種基於毫米波雷達和攝影機融合的人物追蹤系統，並通過一系列實驗對其精密度和性能進行了評估，證明了系統在不同情況下的穩定性和可靠性。該系統能夠有效地對空間中的人物進行追蹤與定位。

In this thesis, we propose LinFusion: a fusion system for indoor human tracking, which integrates millimeter-wave radar and color camera. The millimeter-wave radar is responsible for sensing walking individuals, and can simultaneously measure distance, velocity, and azimuth of multiple targets, obtaining the specific position and speed of people in space. Meanwhile, the color camera side uses object detection and tracking methods in deep learning to obtain the position and unique identifier of people in the image. By fusing the information obtained from these two sensors, the unique identifier and specific position of each person in space can be obtained. To verify the effectiveness of the system, a series of experiments are designed. Millimeter-wave radar and color camera data are collected in an indoor environment, and our fusion system is used to process the data. We conduct experiments on our coordinate calibration method and position estimation method to ensure fusion accuracy. The system's performance evaluation metric is whether the unique identifier of each person changes. We propose a human tracking system based on millimeter-wave radar and color camera fusion, and evaluates its accuracy and performance through a series of experiments, demonstrating the stability and reliability of the system under different situations. The system can effectively track and locate people in space.