OpenAI, a research lab co-founded by Elon Musk, has developed a new breed of AI agents that are capable of playing Dota 2, a complex strategy game, in 5-on-5 multiplayer matches. OpenAI’s new bots have so far been able to beat amateur and semi-professional teams. With this accomplished, the research lab is now looking to bring its bots to The International, a prolific Dota 2 tournament, this coming August.
The new bots go by the name of OpenAI Five, a reference to the number of neural networks working together in the team. To train the neural networks, the AI has been playing roughly 180 years worth of gameplay every day using reinforcement learning. This enables the AI to learn the intricacies of the game, considering that it is far more complicated than board games like Chess and Go. Dota 2, for example, involves hiding data from players, preventing the system from perceiving the entire playing field at a given time.
The hardware employed by the research lab to train OpenAI Five is impressive. The five neural networks train through a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores. The same setup was adopted in a much smaller scale last year when OpenAI rolled out an artificial intelligence system that proved capable of beating the best Dota 2 players in the world in 1-on-1 matches.
Currently, however, OpenAI Five can only play the game with several restrictions. For one, the AI system can only use five of the 115 heroes available in the game. Skills such as Invisibility, Summons, and the placement of wards are also disabled. The research lab, however, hopes that through time, the neural networks would be able to play the game without any restrictions at all.
As could be seen in a recent video shared by the research lab, OpenAI Five is actually being received well by the Dota 2 community. Professional Dota 2 player Blitz, for one, noted that the bots are adopting strategies that are incredibly effective. In a match against OpenAI Five, Blitz, together with four employees of the research lab, put up a fight before getting dominated by the articificial intelligence. In a statement after the game, Blitz sheepishly stated that the bots capitalized on every small error he made during the match.
“I think the team fight aspect of the bot(s) was excellent. It didn’t mess up. When it came to coordination, it was some of the best pure team fighting because it felt like I was getting hammered every single time I made a mistake. I feel like normal humans don’t do that,” the professional Dota 2 player said.
So what’s the secret behind OpenAI Five? In a statement to The Verge, OpenAI CTO Greg Brockman noted that unlike human players, the bots have “no ego” when they play the game. The teamwork aspect of the bots was also trained by allowing them to work individually at first, then encouraging them to work together.
“The bots are totally willing to sacrifice a lane or abandon a hero for the greater good. For fun, we had a human drop in to replace one of the bots. We hadn’t trained them to do anything special, but he said he just felt so well-supported. Anything he wanted, the bots got him,” Brockman said.
Ultimately, Brockman is encouraged by OpenAI Five’s development so far. The research, after all, is motivated by the idea that if AI systems can be trained to perform complex tasks such as learning a game as intricate as Dota 2, it could eventually be used to solve equally complex real-world challenges. Some examples of real-world applications could be designing and managing a city’s transport structure, or the logistics of a massive business.
“This an exciting milestone, and it’s really because it’s about transitioning to real-life applications. If you’ve got a simulation of a problem and you can run it large enough scale, there’s no barrier to what you can do with this,” he said.
