利用深度學習技術自動化決定畫面元素主題

Abstract

在軟體應用程式中，我們會依據畫面上內容的提示而決定我們該在上面執行輸入或點擊之類的操作，然而在進行大量網頁測試的工作時，如何自動化輸入或點擊畫面上的內容成為一個待解決的問題。本篇論文中，我們應用了自然語言處理、深度學習技術。首先根據目標應用程式元素的周圍元素去擷取其特徵，利用Word2Vec、長短期記憶法產生主題向量，藉由比較不同向量之間的相似度我們可以得到期相對應的主題。這些主題可以幫助我們去分析目標網頁元素是否可被執行，藉此來完成自動化測試。

In software applications, users can decide what kind of action they should perform such as fill in text or click elements according to hints on the contents of the screen. However, when doing a lot of application testing, how to automatically fill in text or clicking elements on the screen becomes a problem to be solved. In this thesis, we present a NLP (Natural Language Processing) with deep learning techniques. Firstly, according to surrounding elements of the target element, its feature would be retrieved. Secondly, we transfer the feature into topic vectors using Word2Vec model, Long Short-Term Memory methods. By comparing the similarities between different vectors, we can categorize them to their corresponding topics. These topics can help us tell if a target element can be executed or not, thereby achieving the automatic testing.