利用隨機投影做維度縮減以及探討高斯混合模型

Abstract

隨機投影是在高維度資料分析中縮減維度的方法之一。Johnson-Lindenstrauss 引理闡述在一群高維度的資料投影進入低維度空間中，這些投影資料的相對距離可以維持得很好。換句話說，這群資料的結構沒有因為隨機投影而被破壞到。由於高斯混合模型是最基本被廣泛使用的統計模型之一，所以我們想探討高斯混合模型在隨機投影下的表現。在這篇文章中，我們顯示在某些條件下，高斯混合模型可以通過隨機投影後仍維持著保距性質。然而，投影後資料共變異矩陣特徵值的比例會比起投影前的資料變小。這也許可以解釋高維度資料降維度的分群表現變好。最後，將展示一些關於高斯混合的數值實驗。

Random projection is a promising dimensional reduction technique for high-dimensional data analysis. Johnson-Lindenstrauss Lemma states that a set of points in a high-dimensional space can be embedded into a space of lower dimension in such a way that distances between the points are nearly preserved. In other words, the structure of datasets is not destroyed by random projection. Besides mixtures of Gaussian are among the most fundamental and widely used statistical models. In this article, we show that when a mixtures Gaussians which are separated are projected, the projected Gaussians would be separated through random projection under some conditions. Moreover, the ratio of the eigenvalues of the covariance matrix of projected data becomes little compared with the ratio of the eigenvalues of the covariance matrix of original data. Finally, some numerical experiments with Gaussian mixtures will be illustrated.