改善合成資料以強化CRNN手寫數字辨識之方法

Abstract

深度學習是目前在人工智慧領域中最廣泛被應用的技術，透過網路架構的各種變化，能夠因應各種領域的需求，光學字元辨識（Optical Character Recognition）就是其中一項，而此應用通常稱為文字影像辨識。 文字影像辨識一直是文字資料電子化常用的方法，目前通常透過深度學習模型，來將文字圖片轉換為電子資訊，無論是印刷字還是手寫字，皆可以採用此方法作轉換。 本論文主要透過選用既有資料集EMNIST及68種字體依據EMNIST圖片格式設定之字符圖片，依照所觀察之手寫字符的特徵做強化處理後合成為單一及連續訓練資料，並採用能夠針對連續字符辨識之CRNN模型架構，將資料集分別做單一字符及連續字符的模型訓練。 單一字符實驗結果顯示，透過將手寫字符混入印刷字體能夠達到模型通用性，同一模型對於用來混合之原資料測試集，都能維持或提升辨識水準，而對於手寫表格資料的辨識率，分別由印刷體42%及手寫字82%提升至混合資料集84%。 連續字符實驗結果顯示，在連續字符的合成上加入了所觀察之手寫字符特徵，分別利用漸層強化（向內及向外）模擬手寫字符邊緣的不定性，另外對字符在合成時做大小的隨機縮放及連續字符之間的偏移趨勢（向上、水平及向下），模擬手寫字符的大小及位置隨機性，將手寫表格的辨識率，分別由印刷體5.2632%及手寫字44.211%提升至混合資料集57.895%，其中對3個字符以上的較長字串辨識能力由無法辨識的0%提升至39.3%（11/28），表示特徵強化的必須性。

At present, deep learning is the most widely applied technology in the field of artificial intelligence. Through various changes of network architecture, it can meet the needs of various fields. Optical Character Recognition is one of them. Text image recognition has always been a common method for the digitization of text data. At present, the deep learning model is usually used to convert text images into electronic information, whether printed or handwritten. This thesis mainly through EMNIST optional both data sets and 68 kinds of fonts on the basis of EMNIST image format to set the character image, according to the characteristics of handwritten characters do to strengthen the observation after treatment to synthesize into a single and continuous training data, and can be used against the CRNN model structure of continuous character recognition, the data set, respectively, to do a single character and continuous characters of model training. In this paper, through the choice EMNIST data sets, and chooses 68 kinds of fonts on the basis of EMNIST image format set of character images, according to the observation of the characteristics of handwritten characters to synthesize into a single and continuous training data, and chooses the CRNN model which can use in continuous character recognition, respectively do a single character and continuous characters of model training. The results of single-character experiment show that the model can be universal by mixing handwritten characters into printed fonts, and the recognition level of the same model can be maintained or improved for the test set of original data used for mixing, while the recognition rate of handwritten table data can be increased from 42% in print and 82% in hand to 84% in mixed data sets. Continuous characters, according to the results of the experiment on the synthesis of consecutive characters joined observed characteristics of handwritten characters, respectively, using the gradual layer reinforcement (inward and outward) simulation of handwritten characters on the edge of the uncertainty, in addition to characters in synthetic size when random scaling and the deviation between consecutive characters (upward, horizontal and downward), simulation of handwritten character size and location of randomness. The recognition rate of handwritten forms increased from 5.2632% in printed form and 44.211% in handwritten form to 57.895% in mixed data set, among which the recognition ability of longer strings with more than 3 characters increased from 0%（0/28） to 39.3%（11/28）, indicating the necessity of feature enhancement.