以深度強化學習方式建構混合車流之適應性號誌控制系統

Abstract

台灣之號誌化路口大多以定時號誌為主，但定時號誌可能無法及時反應車流狀況，因此則有應變車流狀態之適應性號誌系統出現，然而在傳統適應性號誌中，許多交通特性之參數方面須由人工假設或校估，進而可能產生人工判斷之誤差。 基於人工智慧之技術逐漸進步，本研究試圖以人工智慧之方式應用於台灣混合車流之號誌化路口，採用人工智慧建構適應性號誌之優勢在於可以減少對於交通特性之參數假設，故在本研究中將採用深度強化學習方式建構出號誌代理幫助適應性號誌進行決策，號誌代理會觀察路口車流環境狀態並做出對應該狀態之最佳動作，本研究首先使用車流模擬軟體VISSIM建立符合台灣車流特性之高混合比號誌化路口車流環境以作為號誌代理學習之環境，並假設號誌代理可觀察路口所有狀態，偵測之狀態特徵包括車輛位置、速率以及當前時相狀態，其中車輛位置及速率是採用格位狀態作為輸入，故本研究以卷積神經網路為基礎預測各動作之價值；而為了使號誌代理學習最佳動作之決策，當號誌代理做出動作之後必須給予其獎勵或懲罰以讓號誌代理往正確之方向學習，本研究採用主要由車輛之停等時間以及路口通過量所組成之加權指標給予號誌代理獎勵或處罰，且另外考量切換時相所產生之紓解時間損失懲罰。最終結果顯示，本研究所建立之適應性號誌在績效表現上皆較固定時制佳，且經過不同情境之測試，驗證本研究之適應性號誌確實具有應變車流變化之能力。

In Taiwan, most of signalized intersection use pretimed control, but pretimed control may not response to traffic conditions in time. Therefore, an adaptive signal control system appears. For traditional adaptive signal control, many parameters of traffic features must be assumed and calibrated artificially. However, these parameters may have errors in artificial assumptions. The study aims to apply artificial intelligence to the signalized intersection of mixed traffic in Taiwan because the advantage of using artificial intelligence to develop an adaptive signal control system is reducing parameter assumptions on traffic features. Hence, the study will develop an agent by deep reinforcement learning which help adaptive signal control system make decisions. In the study, the traffic simulation software VISSIM is used to create virtual environment with high mix rate of motorcycle, and the agent can learn itself on the environment. Assumes that the agent can observe all the state of the environment, the features of state include vehicle position, speed and current phase, and use convolutional neural networks to predict the actions’ value on the specific state. To make the agent learn the best action policy, agent must be rewarded or punished after performing the action because gent’s goal is to maximize a long-term reward. The study defines the reward as weighted sum reward function which contains vehicle’s waiting time, throughput and penalty for changing to the other phase. Finally, the study shows that the adaptive signal control system developed by deep reinforcement learning is better than the fixed time control on the traffic performance. After testing in different scenarios, the study also verifies that the adaptive signal control system can adapt the changes in traffic flow.