透過統計的代理模型對包含量化與類別參數的問題進行自動化參數調校

Abstract

自動化參數調校 (auto-tuning) 的問題在現今的科學計算領域是一個重要 議題。具有龐大運算量的科學計算方法倚靠著先進的電腦計算與模擬， 但由於當前的電腦架構越趨複雜,規模也越來越大, 其中許多的參數須經過調校後，方可獲得最佳計算效率。然而眾多的參數並不容易以手動或自動化的方式進行調校，尤其如果調校參數包含量化與類別變數 (qualitative and quantitative factors)，問題的困難度更高。本研究針對包含量化與類別複合 型態的參數調校問題，提出不同種類的統計代理者模型 (surrogates)、以及不同的循序調校流程，達到自動化參數調校的目的。同時也藉由理論分析與數值實驗，比較分析各個方法的優缺點。再透過這些結果，提出一套模型選擇的方法與準則。

The automatic performance tuning (auto-tuning) problem emerges in recent scientific computing applications. Usually, most of the applications are computationally intensive so that they rely on the computational power of the advanced computer. To achieve better performance, the performance tuning on related factors plays an important role. However, the architecture of modern computer becomes more and more complicated, so that the automatic performance tuning is indispensable. Meanwhile, the related factors involve various types, e.g. quantitative and qualitative factors. The difficulty here is the mixed types of input factors. We studied several statistical approaches (e.g. Gaussian Process model) to deal with such problems. A framework called surrogate-based tuning procedure is proposed, where the surrogate here means a statistical model of the tuning target. Moreover, our tuning procedure is an consecutive procedure, so an effective consecutive tuning procedure is necessary in this framework. To deal with the mixed input types, we proposed a extended method from the classical expected improvement method which is widely used in global optimization problems. And we compare their performances with many testing examples and real data in scientific computing. Finally, based on our results, we concluded a guideline for model selection in the auto-tuning procedure.