UNIVERSITY PARK, Pa. — Researchers in Penn State's College of Information Sciences and Technology are advancing work that utilizes machine learning methods to improve traffic signal control at urban intersections around the world.
Through their work, the researchers are exploring the use of reinforcement learning — training algorithms to learn how to achieve a complex goal through a reward system — to examine patterns and communication of traffic signal control.
“Basically, we want to use traffic data to help us optimize a city’s function,” said Zhenhui "Jessie" Li, associate professor of information sciences and technology.
Li and her collaborators studied real-world traffic data from major cities in the United States and China to analyze intersection congestion. Then, through a series of studies, they developed an in-depth reinforcement learning method that could positively impact traffic flow by learning the best strategies for a traffic flow pattern.
She explained that reinforcement learning is, for example, similar to using positive reinforcement to improve a child’s behavior.
“When teaching kids [using positive reinforcement], if they do a good job you give them some reward,” said Li. “If they do a bad job, then you just don’t give them the reward.”
She continued, “This is the same idea. For a traffic signal, if it does a bad job, cars can get congested and the reward will go down. Then if it’s doing a good job because the cars are saving time at intersections, it will get some reward.”
While Li’s work is focused on analyzing data and the computational aspects of the method, she is working with collaborators across Penn State and at other institutions around the world in the interdisciplinary research.
“As computer scientists, we don’t actually know how to design this reward function,” said Li. “So we’re working with civil engineers to make sure that we use their transportation theory to help us design data.”
She added, “For our setting, the reward would be that the cars take a shorter time to get to their destination. But that is something that is hard to directly quantify because travel time is the result of a long sequence of traffic signals, and it is also affected by other factors such as free moving speed.”
Li collaborated with Vikash Gayah, associate professor in the Department of Civil Engineering, for some of Li's studies on this topic.
Li’s research results have been published in several top conferences in data mining and artificial intelligence in 2019. For example, one paper was presented at the 2019 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, held in Aug. 4-8 in Anchorage, Alaska. Three papers on this work were presented at the 2019 Conference on Information and Knowledge Management, held Nov. 3–7 in Beijing, China. Two other papers have been accepted to the Thirty-Fourth AAAI Conference on Artificial Intelligence, to be held Feb. 7-12, 2020, in New York City.
The major contribution of the latest paper is to scale the traffic signal control into more intersections, according to Chacha Chen, doctoral student in the College of IST and lead author on one of the papers.
“Previously, we’ve typically done experiments on at most 16 intersections,” said Chen. “So the method is not new.”
Li added, “It’s the first time we can scale to one thousand intersections, when it used to be less than 20. It’s 50 times more than [it was] previously, making it truly city-wide.”
The researchers are hopeful that their work could help cities around the world optimize how they use intersections as resources.
“As urbanization continues, and more people move into a city, if you don’t maximize the use of the resource of a road, you would probably end up building more roads to satisfy the needs of the city for the traffic,” said Li. “But the land could be used to build a school or a hospital.
"So the motivation behind our work is to explore if we can use data as a sort of a new resource to help us optimize the use of other resources," she added.
Collaborators on the work include Guanjie Zheng and Hua Wei, doctoral students in the Penn State College of IST; Vikash Gayah, associate professor in Department of Civil Engineering at Penn State; Huichu Zhang, Xinshi Zang, Weinan Zhang and Yanmin Zhu of Shanghai Jiao Tong University; Jie Feng and Yong Li of Tsinghua University; Yuanhao Xiong of Zhejiang University; and Kai Xu of Shanghai Tianrang Intelligent Technology Co. Ltd.
Li’s work is supported by the National Science Foundation and the Haile Family Early Career Professorship from the College of IST.