手寫簽名驗證中自監督學習效果的研究

Abstract

手寫簽名驗證是一種可應用於多種場景的方法，它能夠在銀行或行政機構中用於驗證簽名是否來自於同一作者。因此，它在現代社會中扮演著重要角色。過去的研究中，多採用電腦視覺技術來處理此問題，早期研究通過手工特徵提取方法來獲取簽名特徵，並結合機器學習進行訓練。近年來，研究轉向採用深度學習方法來構建模型並進行訓練，包括應用度量學習和注意力機制。最近，有學者提出使用自監督學習方法來提取簽名特徵，並將其應用於下游任務進行度量學習的手寫簽名驗證框架。我們認為這是一個具有發展潛力的新方向，因此在我們的研究中也採用了這種自監督學習架構來進行手寫簽名驗證訓練。在此過程中，我們探討了不同的資料增強方法對結果的正面或負面影響，同時也利用大型資料集進行自監督學習的預訓練來探討其影響。我們採用Bootstrap Your Own Latent (BYOL) 作為自監督學習模型，並為下游任務構建了一個手寫簽名驗證模型，與其他四種基線方法進行比較。結果顯示，儘管在四種語言資料集：CEDAR（英文）、BHSig-Bengali（孟加拉語）、BHSig-Hindi（印度語）和HanSig（中文）的表現上我們大多排名第三（共五名），但在跨語言資料集的表現上，我們的模型在英文、孟加拉語、印度語上表現最佳，顯示了我們模型的跨語言適應能力，以及使用自監督式學習框架的潛力。

Handwritten signature verification is a method applicable to various scenarios, capable of being used in banks or administrative institutions to verify whether a signature comes from the same author. Hence, it plays a significant role in modern society. In past research, computer vision techniques have been commonly used to address this issue. Early studies extracted signature features using handcrafted methods and combined them with machine learning. In recent years, research has shifted towards using deep learning models, including the application of metric learning and attention mechanisms. Recently, scholars have proposed using self-supervised learning to extract signature features and apply them to downstream tasks for metric learning in handwritten signature verification frameworks. We believe this is a new promising direction; therefore, we adopted this self-supervised learning framework in our study. In this process, we explored how different data augmentation methods could impact the results. We also investigated the impact of pretraining with a large dataset for self-supervised learning. We used Bootstrap Your Own Latent (BYOL) as the self-supervised learning model and constructed a handwritten signature verification model for the downstream task, comparing it with four other baseline methods. The results showed that although we mostly ranked third out of five in performance on the four language datasets: CEDAR (English), BHSig-Bengali (Bengali), BHSig-Hindi (Hindi), and HanSig (Chinese), our model performed best in English, Bengali, and Hindi on cross-language datasets, demonstrating the cross-language adaptation ability of our model and the potential of using a self-supervised learning framework.