A group of researchers from Universitat Politècnica de Catalunya in Barcelona and Huawei have retooled an artificial intelligence technique used for chess and self-driving cars to make OTNs run more efficiently. They will present their research at the upcoming Optical Fiber Conference and Exposition, to be held 3-7 March in San Diego, California, USA.
OTNs require rules for how to divvy up the high amounts of traffic they manage, and writing the rules for making those split-second decisions becomes very complex. If the network gives more space than needed for a voice call, for example, the unused space might have been better put to use ensuring that an end user streaming a video doesn’t get “still buffering” messages. What OTNs need is a better traffic guard.
The researchers’ new approach to this problem combines two machine learning techniques: The first, called reinforcement learning, creates a virtual “agent” that learns through trial and error the particulars of a system to optimise how resources are managed. The second, called deep learning, adds an extra layer of sophistication to the reinforcement-based approach by using so-called neural networks, which are computer learning systems inspired by the human brain, to draw more abstract conclusions from each round of trial and error.
“Deep reinforcement learning has been successfully applied to many fields,” said one of the researchers, Albert Cabellos-Aparicio (pictured). “However, its application to computer networks is very recent. We hope that our paper helps kick-start deep-reinforcement learning in networking and that other researchers propose different and even better approaches.”
So far, the most advanced deep reinforcement learning algorithms have been able to optimise some resource allocation in OTNs, but they become stuck when they run into novel scenarios. The researchers worked to overcome this by varying the manner in which data are presented to the agent.
After learning the OTNs through 5,000 rounds of simulations, the deep reinforcement learning agent directed traffic with 30% greater efficiency than the current state-of-the-art algorithm.
One thing that surprised Cabellos-Aparicio and his team was how easily the new approach was able to learn about the networks after starting out with a blank slate.
“This means that without prior knowledge, a deep reinforcement learning agent can learn how to optimise a network autonomously,” Cabellos-Aparicio said. “This results in optimisation strategies that outperform expert algorithms.”
For more information, visit www.upc.edu/ca