Hopfield 神經網路理論之探討

Abstract

近年人工智慧與神經網路相關的內容遍地開花，相關的理論與應用也不斷地出現在人們的視野裡。我們在Benjamin Scellier 所著的「A deep learning theory for neural networks grounded in physics」[1] 中發現，神經網路也能夠以類似能量的形式來描述並量化其迭代和收斂的過程，因此我們開始對做為其基礎的Hopfield Network 感興趣。 在Hopfield 的概念下[2]，所有神經元的狀態都是二元的，因此神經網路整體可能的狀態數量是有限的。以此為前提，我們便能夠透過能量函數和與其相應的演算法來去判別一個神經網路的收斂性質[3]。在這個基礎上，我們透過Hebbian學習法去建構神經網路，使其能夠確實收斂至事先給定的目標。以手寫數字或車牌辨識為例，我們期望一個辨識系統只會將其判別為指定的英數字或符號，而非其他意料之外的目標。不過在高維度與多個目標下的神經網路其行為過於複雜，因此我們會討論在一些簡化的情況下，是否存在其他我們並不期望的收斂目標。

In recent years, artificial intelligence and neural networks have seen a proliferation of content, as well as the emergence of related theories and applications. In ”A deep learning theory for neural networks grounded in physics” [1] by Benjamin Scellier, we found that neural networks can also describe and quantify their iterative and convergence processes in an energy-like form. Therefore, we become interested in the Hopfield Network as the basis of this theory. In the concept of Hopfield network [2], the states of all neurons are binary, and thus the amount of possible states of the network is limited. Based on this premise, we are able to recognize the convergent property of a neural network by the energy function and its corresponding algorithm [3]. According to this, we use Hebbian learning to construct the neural network so that it can indeed converge to a given target. In the case of handwritten numbers or license plate recognition, for example, we expect a recognition system to recognize only the specified alphanumeric characters or symbols, and not other unexpected targets. However, the behavior of neural networks with high dimension and multiple targets is too complex, so we will discuss whether there are other targets that we do not expect to converge in some simplified cases.