應用深度學習和無人載具整合的多模態方法對櫻桃番茄成熟度進行評估

Abstract

近代農業中，櫻桃番茄（Solanum lycopersicum）是一種經濟重要的農產品。然而，它們的小尺寸和獨特特性，包括短暫的壽命和易受損的脆弱性，給作物監測和收穫帶來了挑戰。迄今為止，大多數先前的研究都集中在開發和優化深度學習模型，用於檢測不同成熟水平的番茄。然而，距離和信心閾值等因素對物體檢測器的檢測性能的影響往往被忽略和未加調查。因此，本研究提出了一種使用無人機（UAV）和YOLOv8自動檢測溫室櫻桃番茄成熟水平的方法。隨後，對上述因素的影響進行了調查。通過使用UDP協議建立了無人機與計算機的通信系統，利用DJI Tello無人機進行了測試，實現了高效的數據傳輸和UAV控制。利用使用DJI Tello、手機和Intel RealSense D435深度相機獲取的櫻桃番茄數據集，對基於YOLOv8n的深度學習模型進行了訓練。微調和消融研究的結果表明，將坐標注意塊和具有動態聚焦機制的邊界框回歸損失（WIoU）損失函數結合起來，可以實現高精度（90.2%）、召回率（88.5%）、F1分數（89.34%）和mAP（93.7%）的成熟檢測模型。開發的模型YOLOv8n + CA + WIoU用於檢測和跟踪，與微調的BoT-SORT跟踪算法一起。結果顯示，BoT-SORT對櫻桃番茄的跟踪效果良好，多目標跟踪精度（MOTA）在74%至87%之間。此外，這些結果進一步證實了低閾值導致更高的敏感性，而高閾值導致更高的特異性和增加的跟踪性能。然而，觀察到YOLOv8n + CA + WIoU算法在40厘米至100厘米的物體至相機距離範圍內開始顯示出檢測確定性下降，特別是在同類番茄之間存在遮擋和接近的情況下。總的來說，這些研究結果突顯了利用UAV系統和先進的深度學習模型在溫室環境中高效準確地監測櫻桃番茄成熟水平的潛力。此外，後續的研究結果還表明，物體至相機距離、信心閾值和遮擋對檢測性能的影響至關重要。解決這些因素對於最大程度地提高UAV基礎的農業監測系統的準確性和可靠性至關重要，進一步加強了這些技術在實際應用和工業應用中的可行性和有效性。

In modern agriculture, cherry tomato (Solanum lycopersicum) is an economically significant commodity. However, their small size and unique characteristics, including their brief lifespan and vulnerability to damage, pose challenges in crop monitoring and harvesting. Up to date, most previous studies focused on developing and optimizing deep learning models for detecting tomatoes at different maturity levels. However, the impact of factors like distance and confidence threshold on the detection performance of object detectors is often ignored and left uninvestigated. Therefore, this study proposed an autonomous method of detecting maturity levels of greenhouse-grown cherry tomatoes using UAV and YOLOv8. Subsequently, the impact of the aforementioned factors was investigated. DJI Tello drone was utilized to setup a UDP-based communication, enabling efficient data transmission and UAV control. For the model training, a cherry tomato dataset comprising images from different modalities were used. The results of fine-tuning and ablation studies demonstrated the effectiveness of incorporating coordinate attention blocks and a bounding box regression loss with dynamic focusing mechanism (WIoU) loss function, achieving high precision (90.2%), recall (88.5%), F1-score (89.34%), and mAP (93.7%) for the ripe detection model. The developed model, YOLOv8n + CA + WIoU, was used for detection and tracking together with a fine-tuned BoT-SORT tracking algorithm. The results revealed the efficacy of BoT-SORT for tracking the cherry tomatoes by achieving Multi-Object Tracking Accuracy (MOTA) ranging from 74 ~ 87% and Counting Accuracy ranging from 60% ~ 85%. Moreover, the results of the study determined the implications that for cherry tomatoes, low threshold (45% ~ 50%) leads to higher sensitivity, while higher threshold (75% ~ 80%) leads to higher specificity and increased tracking performance. However, it was observed that the YOLOv8n + CA + WIoU algorithm started to display decrease in detection certainty at a range of 0.4 m to 1.0 m object-to-camera distance, particularly in the presence of occlusion and close proximity between tomatoes with the same class. The overall findings highlight the potential of using UAV-based systems and advanced deep learning models for efficient and accurate monitoring of the maturity level of cherry tomatoes in greenhouse settings. Furthermore, subsequent findings also demonstrate the critical impact of object-to-camera distance, confidence threshold, and occlusion on detection performance. Addressing these factors are essential for maximizing the accuracy and reliability of UAV-based agricultural monitoring systems, reinforcing the feasibility and effectiveness of these technologies in real-world and industrial applications.