卷積神經網路及深度資訊於柑橘採收狀態評估之應用

Abstract

應用機器視覺協助田間作物偵測，將有助於減緩農業勞動力短缺的困境，並可透過提供即時的生長狀況以利農民制定生產策略。然由於田間狀況多變，如：光線變化、作物與背景間顏色相近、遮蔽等，皆造成傳統影像處理方法之辨識難點。近年來卷積神經網路興起，其具適應性強之優勢，故本研究將應用卷積神經網路進行柑橘作物之辨識及定位，並結合深度攝影機，建立可提供柑橘成熟度、果實數量及尺寸之預測模型。首先採用更快速區域卷積神經網路(faster region convolutional neural network, Faster-RCNN)模型於彩色影像中判讀果實所在位置及其成熟度。成熟度共分為轉色期及成熟期兩類，在設定工作範圍20至200公分內，可得目標物召回率及精確度分別為89%及73%。接續使用支持向量機(support vector machine, SVM)區分出辨識結果中未遮蔽的樣本以進行尺寸評估，將彩色影像辨識出之果實所在區域分別對應至深度圖(depth map)及點雲(point cloud)資訊。取深度圖之平均深度為物距，利用薄透鏡公式(thin lens equation)估計直徑；點雲資訊則進行橢球面擬合，以求得之軸長估計直徑。兩模型之平均絕對百分比誤差(mean absolute percentage error, MAPE)分別為6%及22%。結合以上述流程所開發之柑橘採收狀態評估模型，並配合現有之產銷分級標準，可將影像中之目標物進行初步分級統計。透過所開發之預測模型所提供之果實計數、成熟度及分級分佈之評估等即時資訊，期可協助農民快速掌握作物生長狀態、便利作業安排，及提升田間管理之效率。

Applying machine vision on crops monitoring alleviates the labor shortage issue, and it helps framers to formulate production strategies by providing real-time growing status. However, various field conditions pose a higher challenge in traditional image processing, such as illumination changes, color similarity, and occlusion, etc. In recent years, convolutional neural networks have gradually risen, and it has the advantage of strong adaptability. Therefore, this study applied the convolutional neural network to identify and locate citrus fruits, and it combined a depth camera to establish a predictive model. It could provide the information of ripeness, and estimate the fruit counts and associated sizes. First, faster region convolutional neural network (Faster RCNN) was applied to the color images to localize the fruit position and identify two ripeness, including ripening and ripened. The recall and precision of all fruits were 89% and 73% in the measuring range from 20 to 200 cm. Then the support vector machine (SVM) were applied to distinguish the non-occluded samples in the identification results for size estimation. The identified regions in the color image were mapped to the depth map and point cloud, respectively. The object distances were the average of depth maps, and the thin lens equation was applied to obtain the estimated diameters. The point clouds were conducted with ellipsoid fitting method to obtain the estimated diameters according to the resulted major axis length. The mean absolute percentage error (MAPE) of these two models were 6% and 22%, respectively. The proposed algorithm, evaluating the harvesting status for citrus fruit, was obtained by combining the above-mentioned processes. Cooperating with the existing grading standards, the objects in the images can be classified and counted. Through the developed model, real-time information including the fruit counts, stages of ripeness, and distribution of the potential grading could be provided. It helps farmers to monitor the growth condition in time, facilitates operation arrangement, and improve the efficiency of field management.