Researchers at Facebook have developed a general AI framework called Recursive Belief-based Learning (ReBeL) that they believe achieves better than human performance in heads-up, no-limit Texas Hold’em while using less domain knowledge than any poker AI made before. The team believes that ReBeL is a step towards developing universal techniques for multi-agent interactions, in other words, general algorithms that can be used in different fields. Potential applications run the gamut from auctions, negotiations, and cybersecurity to self-driving cars and trucks.
The combination of reinforcement learning with search at AI model training and test time has led to a number of significant advancements.
“Reinforcement learning is where agents learn to achieve goals by maximizing rewards, while search is the process of navigating from a start to a goal state.”
For example, DeepMind’s AlphaZero employed reinforcement learning and search to achieve impeccable performance results in board games like chess, shogi, and Go. However, this approach hits a snag when applied to imperfect-information games like poker because it makes a number of assumptions that don’t hold in these scenarios. The value of any given action depends on the probability that it’s chosen, and more generally, on the entire play strategy.
The Facebook research team proposes that ReBeL offers a fix. ReBeL builds on work in which the notion of “game state” is expanded to include the agents’ belief about what state they might be in, based on common knowledge and the policies of other agents. ReBeL trains two AI models — a value network and a policy network — for the states through self-play reinforcement learning. It uses both models for search during self-play. The resulting model is a simple, flexible algorithm that the researchers claim is capable of defeating top human players at large-scale, two-player imperfect-information games.
Also Read: Advanced Poker Strategies And Tips
ReBeL could lead to a game-changing series of events in different fields of gaming if Facebook is able to create it to the required level.