Reaction to 'Eco-Driving measures could significantly reduce vehicle emissions' by MIT News

MIT News published an article on August 7th detailing a methodology undertaken by researchers that has the potential to reduce global vehicle carbon emissions. The base of this methodology was achieved via the usage of one of the most important deep learning methods in the ever-growing field of machine learning: deep reinforcement learning.

On a high level, reinforcement learning is a sequential method in machine learning where a specific agent is placed in an environment (similar to a character in a video game), and the agent takes steps and tries different things to maximize its 'reward' or success. Essentially, an agent will learn from its mistakes in the environment and develop an optimal strategy known as the policy, which allows it to solve the objective problem posed by the environment. Deep reinforcement learning is analogous to standard reinforcement learning as is deep learning analogous to standard machine learning; it is more complex and involves more layers for increased performance.

In the article, researchers used deep reinforcement learning to stimulate and model as a 'decentralized cooperative multi-agent control problem'. Within this model, they were able to generate eco-driving measures with the potential to reduce CO2 emissions in areas of busy traffic. The model was focused on three specific cities in the United States, and led to the discovery that undergoing certain eco-driving measures such as dynamically optimizing speed limits can cut such pollution by almost 10%.

How was this model formulated? Using a series of factors (33) that were believed to influence emissions, including but not limited to 'temperature, road grade, intersection topology, age of the vehicle', the researchers combined data on these factors with data from OpenStreetMap, U.S. geological surveys, and other databases to create 'digital replicas' of a number of signalized intersections in Atlanta, San Francisco, and Los Angeles. They then employed deep reinforcement learning to optimize each scenario portrayed by these replicas using a 'high-fidenlity traffic stimulator', intuitively assigning rewards to actions that reduced emissions and penalties to those that increased them.

A key characteristic for model simplification the researchers chose to hold was that they implemented each vehicle in the stimulation in a decentralized manner; in other words, each vehicle cooperated with each other in simulation to minimize energy usage. However, because this proved difficult to generalize across all types of traffic scenarios and environments in reality, the researchers chose to train multiple models in parallel, on different clusters of traffic scenarios. Due to the computational cost, the lab ultimately decided the best way to progress in each model was to analyze at the intersection level.

Despite some critical remarks on the applicability of the analysis derived from these training scenarios to real-life situations, where unpredictable human behaviors and unforseen factors may emerge, this methodology provides a solid base for deriving green solutions to our environment with the use of machine learning. In particular, I find its use of reinforcement learning fits well with the task at hand, given that a vehicle typically takes a sequence of steps/actions relative to the other agents in the model.

However, I am curious about the training process the researchers used to reach their optimal solutions for eco-driving. They seemed to train on multiple models at once, each comprising of its own 'cluster of of traffic scenarios'. How did classify these clusters, or at least determine which cluster a particular scenario they derived from their data belong to? Did they run a clustering algorithm under the hood to derive these categories? This is important for us to know, as is typical in any training of a machine learning model's data preprocessing. Another question related to their training process: it seems evident that they trained their models using some data they obtained from various sources. However, it seems that the testing process was not thorough, or at the very least, the 'test'/'validation' was on actual scenarios. Could the researchers have done more in the validation or testing phases of the training to improve their analysis or make their results more viable to match conditions in the real world? The researchers themselves would be wary that they may not have accounted for all influential factors impacting traffic behaviors, including some more unobservable metrics such as human behavior, as they acknowledged themselves in the article. I feel that if a more rigorous testing or re-training is performed on the results currently derived from the reinforcement models, which could include the addition of some bias factor or constraints on the policy, the analysis would be more validating.