從生成對抗網路轉換活體皮膚斷層影像為類H&E染色影像

Abstract

在臨床上，有關皮膚方面的疾病，病理學家在判斷上主要還是依靠染色切片作為黃金標準，其中又以蘇木精與伊紅染色(H&E staining)切片最為常見，但因為其製備過程的破壞性與費時，在手術上常常會造成延誤手術時間、多餘組織被破壞等缺點。而光學同調斷層掃瞄術(OCT)所展示的皮膚三維影像具有高解析度與非侵入式等特性，能夠在不破壞組織的情況下，提供皮膚的完整精細結構，然而因OCT影像對於病理學家來說並不熟悉，因此在臨床上儘管有了OCT影像，病理學家仍然會製備染色切片來做更精確的診斷，顯示出了染色影像的必要性。因此本研究為了使病理學家能夠更直觀的利用OCT影像做診斷，建立了能夠將OCT影像轉換為類H&E染色影像的模型，稱為OCT2HE模型。 基於使用活體人類皮膚的OCT影像轉換為類H&E染色影像之目的，在影像轉換的領域上，生成對抗網路(GAN)相較於其餘的網路架構具有能夠使用非成對影像(Unpaired data)作訓練的潛力，而這對於本研究之目的也是最關鍵的，因為無法從活體組織中取得染色影像，因此本研究所提出的影像轉換模型─OCT2HE模型，其建立在以GAN為核心的CycleGAN上，並針對活體人類皮膚的OCT影像與人類皮膚的H&E影像做非成對影像的訓練，透過改善了在訓練過程中所碰到的雜訊重建問題、皮膚角質層(SC)下邊界與表皮真皮交界(DEJ)轉換上與醫生所判定的標準答案不相符的問題，以及細胞核位置轉換前後對應不正確的問題，成功達到了使活體人類皮膚的OCT影像轉換為類H&E影像的目標，並且提供了量化數值方便比對。目前的成果為在測試影像上，SC下邊界於模型輸出與輸入影像的位置誤差為±1.74 μm，DEJ於模型輸出與輸入影像的位置誤差為±2.29 μm；針對SC下邊界至DEJ間的區域，此區域之IOU於模型輸出與輸入影像為87%±3%，以及此區域的皮爾森相關係數於模型輸出與輸入影像為87%±1%。 本論文利用OCT2HE模型可將活體人類皮膚的OCT影像轉換為近似的H&E染色影像，轉換後的影像不僅針對醫生所判定的OCT影像之SC下邊界與DEJ有滿高的一致性，就連比較細節資訊的細胞核也能大致正確的轉換。此OCT2HE模型利用OCT影像的非侵入式特性與影像轉換技術的即時轉換特性，有望能改善目前臨床上染色影像所帶來的費時與必需破壞活體組織等缺點。

For pathologists, histopathological analysis like hematoxylin and eosin dye (H&E) staining is considered as the gold standard for skin diagnosis. However, the long staining processing time and destroying the tissue cause pathologists inconvenience especially when surgery is progressing. On the other hand, optical coherence tomography (OCT) enables non-invasive, high resolution and 3-D imaging of the structure of skin tissue. Owing to the huge difference between OCT images and H&E images, pathologists are hard to diagnose disease with OCT images only. It shows the necessary of stained images when diagnosis. In this research, we proposed a semi-supervised image-to-image translation model, called OCT2HE model. This model depends on the CycleGAN, but improve some problems for training on in vivo human skin OCT images and human skin H&E images. First, we solved the problem of noise reconstruction in training. Second, we improved the problem of the stratum corneum (SC) lower boundary and dermal-epidermal junction (DEJ) mismatched to the ground truth after translation. Third, we even improve the location of nuclei before-and-after translation. Here are some quantity values to show how accurate for the translation. For the SC lower boundary, the error between input and output of the OCT2HE is ±1.74 μm. For the DEJ, the error between input and output is ±2.29 μm. For the IOU in the region between SC lower boundary to DEJ, the value is 0.87±0.03. For the Pearson’s correlation coefficient in the region between SC lower boundary to DEJ, the value is 0.87±0.01. Our OCT2HE model provides a real time image-to-image translation between OCT and H&E stained-liked images. If combing this model with OCT technology, it could be very help for the pathologist when surgery is progressing, and it could solve the problems of H&E staining processing nowadays.