基於深度學習之番茄表型紀錄與成熟度預測系統開發

Abstract

番茄(Solanum lycopersicum)作為全球重要的蔬菜作物，是人類日常飲食中維生素和茄紅素的重要來源，但氣候變遷、病原體擴散及勞動力短缺等挑戰威脅其生產穩定性。而番茄的育種研究以及自動化採收工具的使用可以有效解決以上問題。因此，本研究旨在開發兩套基於深度學習與影像處理的系統，分別協助育種研究進程以及自動化採收工具開發：用於田間表型評估的番茄時序觀測與監測系統(TOMATO)以及用於收穫評估的番茄成熟度預測系統(TRPS)。 TOMATO針對大果番茄進行自動化田間表型評估，包含植株定位系統、性狀識別模型、表型分析、資料庫系統四大功能模組。系統運作如下: 將包含多株植株的影片上傳至系統資料庫後，植株定位系統從影片中自動提取個別植株影像，其結合Marigold單目深度估測技術與Transformer時序建模架構。接著性狀識別模型對提取的番茄植株影像進行四種性狀: “花”、 “綠果”、 “轉色果”和“採收果”的識別，並設計二階段識別策略提高模型表現，模型採用DINO (DETR with Improved deNoising anchOr boxes)作為主架構，並利用EfficientNetV2對低信心分數的物件進行重分類。性狀識別結果會按照所屬植株及記錄日期系統化的儲存。進行表型評估時會從資料庫提取目標基因型的識別結果，並根據表型判定標準分析始花期、始果期及轉色期三種關鍵表型指標。本系統的實驗資料蒐集自位於臺灣臺南的世界蔬菜中心。TOMATO的植株提取表現在實驗中達到91.9%的提取準確率，每株植物處理時間15.9秒。性狀識別模型的平均精度均值(mean Average Precision, mAP)達到0.862，F1分數(F1 score)最高達到0.826。使用包含44個基因型的完整生長季資料測試，關鍵期的預測平均絕對誤差(Mean Absolute Error)為4.3天。最後為提高TOMATO的使用方便性開發網頁介面，支援遠端監控與自動化資料處理，提升番茄育種研究效率，減少人力負擔。 TRPS針對玉女番茄量化的成熟度評估，包含果實檢測、紅熟指數預測及糖度預測三部分組成。系統運作如下: 採用You Only Look Once version 11 (YOLOv11)為架構的果實偵測模型偵測影像中的果實物件，果實包含“綠果”、 “轉色果”和“採收果”三種成熟度類別。接著針對“採收果”類別透過基於HSV色彩空間的紅熟指數(Red Ripeness Index)量化演算法，將果實成熟度以60到100的連續數值評估。為克服光照變化影響，採用Perfect Reflection Algorithm及多尺度Retinex色彩復原(MSRCR)技術對影像進行色彩校正試驗。糖度預測採用分布密度加權方法建立RRI-糖度分類規則，將果實根據糖度分為四個等級。本系統實驗資料蒐集自台北、桃園、新竹的溫室。TRPS的偵測模型的平均精度均值(mAP@0.5)達93.1%，其中“採收果”偵測精確度(Precision)可達92.1%。RRI與實測色調值呈強負相關(溫室環境r=-0.952)，在不同光照條件下展現良好穩定性。糖度預測準確率為52%，可作為收穫決策的輔助參考。TRPS系統能夠在複雜溫室環境中提供準確的果實紅熟度量化評估，為自動化採收工具提供收穫時機判斷依據，以提升採收效率並優化果實品質管理。

Tomato (Solanum lycopersicum), as a globally important vegetable crop, serves as a vital source of vitamins and lycopene in the human diet. However, challenges such as climate change, pathogen proliferation, and labor shortages threaten its production stability. Tomato breeding research and the implementation of automated harvesting tools can effectively address these issues. Therefore, this study aims to develop two systems based on deep learning and image processing to assist breeding research progress and automated harvesting tool development: the Temporal Observation and Monitoring system for Automated Tomato phenOmics (TOMATO) for field phenotypic evaluation and the Tomato Ripeness Prediction System (TRPS) for harvest assessment. TOMATO conducts automated field phenotypic evaluation for large-fruit tomatoes, comprising four functional modules: Plant Positioning System, trait identification model, phenotypic analysis, and database system. The system operates as follows: after uploading videos containing multiple plants to the system database, the Plant Positioning System automatically extracts individual plant images from videos, combining monocular depth estimation method with Transformer architecture. Subsequently, the trait identification model identifies four traits in the extracted tomato plant images: “Flower,” “Green stage fruit,” “Turning-Pink stage fruit,” and “Harvest stage fruit,” implementing a two-stage identification strategy to improve model performance. The model employs DINO (DETR with Improved deNoising anchOr boxes) as the main architecture and utilizes EfficientNetV2 for reclassification of low-confidence objects. Identification results are systematically stored according to their corresponding plants and recording dates. During phenotypic evaluation, identification results of target genotypes are extracted from the database, and three critical phenotypic indicators—flowering date, fruiting date, and color break date—are analyzed according to phenotypic determination criteria. Experimental data for this system were collected from the World Vegetable Center located in Tainan, Taiwan. TOMATO's plant extraction performance achieved 91.9% extraction accuracy with a processing time of 15.9 seconds per plant. The trait identification model achieved a mean Average Precision (mAP) of 0.862 and a maximum F1 score of 0.826. Testing with complete growing season data containing 44 genotypes showed a Mean Absolute Error (MAE) of 4.3 days for critical date prediction. Finally, a web interface was developed to enhance TOMATO's usability, supporting remote monitoring and automated data processing, improving tomato breeding research efficiency and reducing labor burden. TRPS provides quantitative ripeness prediction for cherry tomatoes, comprising three components: fruit detection, Red Ripeness Index (RRI) prediction, and Brix prediction. The system operates as follows: a fruit detection model based on You Only Look Once version 11 (YOLOv11) architecture detects fruit objects in images, including three ripeness categories: “Green stage fruit,” “Turning-Pink stage fruit,” and “Harvest stage fruit.” For “Harvest stage fruit”, the HSV color space-based Red Ripeness Index (RRI) quantification algorithm evaluates fruit ripeness using continuous numerical values ranging from 60 to 100. To overcome illumination variation effects, Perfect Reflection Algorithm and Multi-Scale Retinex with Color Restoration (MSRCR) techniques were employed for image color correction experiments. Brix prediction utilizes a distribution density weighting method to establish RRI-brix classification rules, categorizing fruits into four levels based on brix content. Experimental data for this system were collected from greenhouses in Taipei, Taoyuan, and Hsinchu. TRPS detection model achieved 93.1% mAP@50 performance, with “Harvest stage fruit” detection precision reaching 92.1%. RRI showed strong negative correlation with measured hue values (r=-0.952), demonstrating good stability under different illumination conditions. Brix prediction accuracy was 52%, serving as auxiliary reference for harvest decisions. TRPS system provides accurate red ripeness prediction in greenhouse environments, offering harvest timing criteria for automated harvesting tools to improve efficiency and quality management.