利用Transformer機器學習模型進行性質預測以及分子設計

Abstract

本研究探索將計算化學(computational chemistry)結合機器學習(machine learning)技術的可能性。在特用化學品的開發以及藥物設計的領域，描述化學結構以及化學性質間的關係一直扮演著很關鍵的角色。傳統上可以在實驗室內透過進行大量實驗的試誤法(trial and error)來達成此目的，但這個方法的效率不彰，且需要大量的時間、人力、金錢等資源成本。因此在這個研究中，我們嘗試利用電腦計算模擬來幫助我們。我們先使用與自然語言(natural language)相近的SMILES(simplified molecular input line entry specification)表示法，讓電腦能夠了解分子的結構，接著利用目前最新穎的自然語言處理模型，也就是Transformer模型，並結合變分自動編碼器(variational autoencoder)的概念來達成性質預測以及分子設計的任務。 在這個方法背後的假設是我們可以利用一個想像出來的高維度向量空間，來逼近實際上複雜的化學空間。因此我們利用模型可應用的三種場景來驗證這個假設。這些使用情境分別是隨機的生成大量新分子，預測感興趣分子的性質，以及設計出具有目標性質的分子。首先，我們發現模型針對訓練分子的重建精確度(reconstruction accuracy)以及設計出新分子的有效性(validity)存在著權衡的關係(trade-off)。接著，我們發現除了訓練資料集內的分子外，模型同樣可以預測新產生分子的性質，這代表著模型的泛化(generalization)能力。最後，我們成功同時利用模型的生成能力以及預測性質的能力，來設計出具有目標性質的分子。總而言之，我們展示了計算化學以及機器學習技術之間的結合，可以同時在性質預測以及分子設計這兩個任務上達到優秀的表現。

This work explores the possibility at the nexus of computational chemistry and machine learning. Describing the relationship between the chemical structure and chemical property has always played an important role in drug design and specialty chemicals development. The most intuitive way is the trial and error approach in the laboratory, but they are inefficient and resource-intensive. Therefore, this work adopts computer-aided approaches. We use language-like SMILES notations to represent the molecular structure on the computer, then leverage the state-of-the-art natural language processing model, that is, Transformer, and integrate the concept of the variational autoencoder to perform property prediction and molecular design. The hypothesis underpinning this work is that we can use a high-dimensional vector space to approximate the complicated real chemical space. Therefore, we verify this hypothesis from three application scenarios: randomly generating a large number of new molecules, predicting the property of molecules of interest, and designing new molecules with the targeted property. First, we prove there is a trade-off between the reconstruction accuracy of the training data and the validity of the newly generated molecules. Besides, the model is not only limited to the training data but can also predict the property of the newly generated molecules, which represents the generalization ability of the model. Finally, we successfully utilize the generative ability and property prediction capability of this model to design molecules with the targeted property. To sum up, we demonstrate that the convergence between computational chemistry and machine learning can create unprecedented performance on both property prediction and molecular design simultaneously.