Institute of Transportation, MOTC

         In recent years, due to the rapid development of artificial intelligence in software and hardware technology and the rapid development of applications in various fields, technologies such as artificial intelligence, image recognition, information andcommunication technologies (ICT), vehicle-to-everything (V2X) and 5G can be expected to be used in the future as they especially alleviate pain points caused by the lack of intelligence in traffic control signals. In addition to reviewing and collecting the applications and algorithms of the Internet of Vehicles and artificial intelligence reinforcement learning in signal control both at home and abroad, this project also develops a simulated environment for signal control of the Internet of Vehicles and artificial intelligence reinforcement learning. Artificial intelligence signal control model construction, learning and training, simulation, and performance evaluation of multi-temporal signal control at a single intersection and arterial chain signal control simulation can lay the foundation for subsequent research and development of artificial intelligence signal control. The plan is to use three main roads at the intersection of "Zhongshan North Road-Dexing East Road" in Taipei City, the intersection of "Taiwan 86-19A" in Tainan City, and the intersection of Tai 88 Fengshan Exit (Guopi Road-Fengding Road) in Kaohsiung City) etc. where the collaboration between county and city governments led
to the development of a traffic simulation environment, as well as artificial intelligence reinforcement learning signal control model and its training.
        According to the characteristics of different scenarios and the topics discussed in reinforcement learning, a total of 13 different reinforcement learning application schemes are proposed. The training results show that all 13 reinforcement learning schemes can improve current traffic flow to varying degrees, this showcasing the potential and feasibility of control through DDPG reinforcement
learning. At the same time, the plan is to develop the method of using PPO as the core algorithm and conduct a simulation with Tainan City as a testbed. The results showed that different reinforcement learning algorithms can also achieve improvements. In the application of Internet of Vehicles data, the research team uses a convolutional layer network (CNN) structure to extract data features, and then use it as data input for reinforcement learning. The experiment results show the potential of using Internet of Vehicles data as a source of reinforcement learning signal control data.