基於深度強化學習將動態市場條件納入投資組合最佳化

Abstract

本研究探討在動態且複雜的金融市場中優化投資組合策略的挑戰。傳統方法如Markowitz均值-方差模型，由於其靜態性和對歷史數據的依賴，在實際應用中往往效果不佳。深度強化學習（Deep Reinforcement Learning）模型，特別是DeepTrader，已顯示出適應市場變化的潛力。然而，這些模型在應用於不同市場（如台灣市場）時，往往會出現極端表現變化的問題。本研究旨在通過整合Transformer網絡來改進DeepTrader模型，替換其原始模型中的Graph Convolutional Network和Long Short-Term Memory部分。Transformer網絡因其優越的長期依賴關係捕捉能力和對股票間複雜關聯的處理能力而被選中。我們在美國和台灣市場上測試了所提出的改進模型，以評估其性能和穩定性。具體實驗包括在美國市場上運用DeepTrader模型，深入分析其在不同市場條件下的行為和性能表現，並比較DeepTrader模型在台灣市場上的表現，識別其在長期關聯學習中的不足之處。此外，我們測試改進後的Transformer模型，評估其在穩定性和適應不同市場環境方面的提升。研究結果表明，改進後的模型在處理市場波動和適應不同市場環境方面顯著優於原始DeepTrader模型，特別是在應對市場變化和捕捉長期依賴關係方面顯示出更強的能力。每月更新權重又可以將改進後的Transformer模型效果發揮得更好，使其在動態市場環境中保持更高的穩定性和準確性。

This research addresses the challenges of optimizing portfolio strategies in dynamic financial markets. Traditional methods like the Markowitz mean-variance model often fail due to their static nature. Deep reinforcement learning (DRL) models, particularly DeepTrader, show promise but struggle with performance variability across different markets, such as Taiwan. We aim to enhance DeepTrader by integrating Transformer networks, replacing the Graph Convolutional Network and Long Short-Term Memory components. Transformers were chosen for their ability to capture long-term dependencies and handle complex stock relationships. We tested the improved model in both U.S. and Taiwanese markets. Key findings include analyzing DeepTrader's behavior in the U.S. market under various conditions, identifying performance issues in the Taiwanese market related to long-term correlation learning, and demonstrating that the Transformer-based model improves stability and adaptability across different markets. The improved model significantly outperforms the original DeepTrader in handling market volatility and adapting to diverse market environments, showing stronger capabilities in capturing long-term dependencies and responding to market changes. Additionally, implementing monthly updates to the model's weights further enhances the performance of the Transformer-based model, ensuring greater stability and accuracy in dynamic market environments.