鯨豚哨叫聲自動化偵測方法之性能評估研究

Abstract

本研究旨在針對鯨豚哨叫聲自動化偵測方法建立一套標準化且易於實作的性能評估流程，以解决不同偵測方法之性能比較不易的問題。本研究提出兩種基於頻譜偵測概念的匹配流程，作為量化偵測性能的統一評估架構。 研究錄音資料取自臺灣西部彰濱工業區及線西水道鄰近海域之豐富哨叫聲時段，比較三種偵測方法（NTUPAM、GM-PHD與PAMGuard）在時間網格匹配（TGM）與輪廓匹配（CM）下的偵測表現。TGM衡量哨叫聲是否真實存在的判定能力，CM用於衡量輪廓追蹤能力，兩流程採用tLimit、tgroup、SNRthr、dfthr與ratiodf等匹配閾值探討其對偵測結果的影響，以精確率（Precision）、召回率（Recall）與F1分數（F1-score）作為性能評估指標，並依據閾值設定自動排除不可納入評估計算的資料。結果顯示，提升tLimit或SNRthr會降低可評估樣本比例使Precision下降；提升tgroup可減少漏檢使Recall提升；降低ratiodf則可減輕局部離群輪廓點的影響，使更多輪廓通過dfthr檢核提升整體偵測性能。在偵測方法比較上，NTUPAM於TGM中有最高Precision（0.8660），較不易出現真實哨叫聲的誤判（FP），但於輪廓追蹤表現較差；GM-PHD於TGM與CM皆有最高Recall（0.9273；0.8669），顯示其具有良好輪廓追蹤性能，且最能避免漏檢（FN）；PAMGuard則於兩流程均獲最高F1-score（0.8668；0.8508），呈現整體均衡的偵測性能。此外，本研究發現部分性能指標（如Accuracy、Error Rate）在長期尺度資料下易受高佔比之TN主導，不一定可有效反應偵測性能，應釐清分析時段與資料規模對此類指標表現的潛在影響。 本研究所建置的性能評估流程具備彈性與可擴充性，未來可納入更多場域資料與偵測方法，使匹配流程更趨完善，並整合至自動化偵測平台，以期建立更具普適性與實務價值的鯨豚聲學偵測性能比較框架。

This study aims to establish a standardized and easily implementable performance evaluation process for automatic cetacean whistle detection methods, addressing the difficulty of comparing detection performance across different algorithms. Two matching procedures based on spectrogram-based concepts are proposed as a unified framework for quantifying detection performance. The analyzed acoustic recordings were collected from whistle-rich periods in the coastal waters near the Changbin Industrial Park and the Xianxi Channel in western Taiwan. Three detection methods—NTUPAM, GM-PHD, and PAMGuard were compared under two matching procedures: Time-Grid Matching (TGM) and Contour Matching (CM). TGM evaluates the detector’s ability to determine the actual presence of whistles, while CM focuses on contour-tracking performance. Both procedures employ matching thresholds including tLimit, tgroup, SNRthr, dfthr and ratiodf to examine their influence on detection outcomes. Precision, Recall, and F1-score are used as performance metrics, and data that do not meet evaluation conditions are automatically excluded based on threshold settings. Results show that increasing tLimit or SNRthr reduces the proportion of evaluable samples, leading to decreased Precision; increasing tgroup reduces missed detections and thus improves Recall; lowering ratiodf mitigates the influence of local outlier contour points, allowing more contours to pass the dfthr criterion and improving overall detection performance. In comparing the three detection methods, NTUPAM achieved the highest Precision (0.8660) in TGM, indicating fewer false positives for true whistle occurrences, though its contour-tracking performance was relatively poor. GM-PHD achieved the highest Recall in both TGM and CM (0.9273; 0.8669), suggesting strong contour-tracking ability and minimal missed detections (FN). PAMGuard obtained the highest F1-score in both procedures (0.8668; 0.8508), demonstrating balanced overall performance. Additionally, this study found that certain metrics (such as Accuracy and Error Rate) are easily dominated by the high proportion of TN in long-duration datasets, which may not effectively reflect detection performance; therefore, the potential influence of analysis duration and data scale on such metrics should be carefully examined. The performance evaluation process developed in this study possesses flexibility and extensibility. Future work may incorporate additional datasets from various environments and a broader range of detection methods to further refine the matching procedures and integrate them into automated detection platforms, with the goal of establishing a more universally applicable and practically valuable framework for cetacean acoustic detection performance assessment.