Vocal2Fail：演奏不佳程度可控的人聲至破音直笛風格轉換

Abstract

音色轉換的目標是在保留輸入音訊內容的同時，改變其音色。在本研究中，我們深入探討「失敗音樂音色風格轉換」（failed music timbre transfer）任務，並開發一套能夠進行歌聲轉換為「破音直笛」的音色轉換系統，藉由屬性向量（attribute vector）實現對演奏失敗程度的可控調節。為了解決音色轉換領域中，特別是在「失敗音樂」情境下缺乏客觀評估標準的問題，我們引入一組客觀評估指標：用以捕捉病態聲音特徵的諧波噪音比（Harmonics-to-Noise Ratio, HNR）、用以衡量音高輪廓一致性的動態時間校正距離（Dynamic Time Warping, DTW），以及根據哼唱選歌（Query by Singing/Humming, QbSH）設計的旋律辨識度指標。實驗結果顯示，這些指標與人類感知高度一致，能有效反映可控的演奏失敗程度。我們的研究為音色轉換任務中的表現劣化評估與控制提供了穩健的基礎。

The goal of timbre transfer is to modify the timbre of an input audio while preserving its content. In this work, we conduct an in-depth investigation into the failed music timbre transfer by developing a vocal-to-failed-recorder timbre transfer system with an attribute vector for poor performance controllability. To address the lack of objective evaluation criteria in timbre transfer, particularly for failed music, we introduce a set of objective metrics: Harmonics-to-Noise Ratio (HNR) for capturing pathological sound traits, Dynamic Time Warping (DTW) distance for assessing pitch contour consistency, and Query by Singing/Humming (QbSH)-based metrics for quantifying melodic identity preservation. Experiments show these metrics align well with human perception and effectively reflect controllable poor performance. Our work offers a robust foundation for evaluating and controlling performance degradation in timbre transfer tasks.