Man to battle machine in poker matchup

No ‘magic recipe’
The game-tree approach doesn’t work in poker because in many situations there is no one best move. There isn’t even a best strategy. Top-notch players adapt their play over time, exploiting their opponent’s behavior. They bluff against the timid and proceed cautiously when players who raise only on the strongest hands are betting the limit. They learn how to vary their own strategy so others can’t take advantage.

That kind of insight is very hard to program into a computer. You can’t just give the machine some rules to follow, because any reasonably competent human player will quickly intuit what the computer is going to do in various situations.

“What makes poker interesting is that there is not a magic recipe,” Schaeffer said.

In fact, the simplest poker-playing programs fail because they are just a recipe, a set of rules telling the computer what to do based on the strength of its hand. A savvy opponent can soon gauge what cards the computer is holding based on how aggressively it is betting.

That’s how Laak was able to defeat a program called Poker Probot in a contest two years ago in Las Vegas. As the match progressed Laak correctly intuited that the computer was playing a consistently aggressive game, and capitalized on that observation by adapting his own play.

Game theory enters into the fray
Programmers can eliminate some of that weakness with game theory, a branch of mathematics pioneered by John von Neumann, who also helped develop the hydrogen bomb. In 1950 mathematician John Nash, whose life inspired the movie “A Brilliant Mind,” showed that in certain games there is a set of strategies such that every player’s return is maximized and no player would benefit from switching to a different strategy.

In the simple game “Rock, Paper, Scissors,” for example, the best strategy is to randomly select each of the options an equal proportion of the time. If any player diverted from that strategy by following a pattern or favoring one option over, the others would soon notice and adapt their own play to take advantage of it.

Texas Hold ’em is a little more complicated than “Rock, Paper, Scissors,” but Nash’s math still applies. With game theory, computers know to vary their play so an opponent has a hard time figuring out whether they are bluffing or employing some other strategy.

Winning vs. non-losing
But game theory has inherent limits. In Nash equilibrium terms, success doesn’t mean winning — it means not losing.

“You basically compute a formula that can at least break even in the long run, no matter what your opponent does,” Billings said.

That’s about where the best poker programs are today. Though the best game theory-based programs can usually hold their own against world-class human poker players, they aren’t good enough to win big consistently.

Squeezing that extra bit of performance out of a computer requires combining the sheer mathematical power of game theory with the ability to observe an opponent’s play and adapt to it. Many legendary poker players do that by being experts of human nature. They quickly learn the tics, gestures and other “tells” that reveal exactly what another player is up to.

Modeling the game
A computer can’t detect those, but it can keep track of how an opponent plays the game. It can observe how often an opponent tries to bluff with a weak hand, and how often she folds. Then the computer can take that information and incorporate it into the calculations that guide its own game.

“The notion of forming some sort of model of what another player is like ... is a really important problem,” Nau said.

Computer scientists are only just beginning to incorporate that ability into their programs; days before their contest with Laak and Eslami, the University of Alberta researchers are still trying to tweak their program’s adaptive elements. Billings will say only this about what the humans have in store: “They will be guaranteed to be seeing a lot of different styles.”

Even so, Laak and Eslami are top-notch players with a deep understanding of poker’s mathematical fundamentals. They should be able to keep up with the computer — this time.

MORE FROM INNOVATION

Innovation Section Front

Sticky glue out-geckos the geckos Plastic film could make house lights obsolete Robot suit for rent in Japan to help people walk Lying about your age? A computer can tell Power cell wins $1M Pentagon prize Scientists develop solar cells with a twist Cleaner way to drink from the seas? Cows may hold secret for efficient ethanol 'Gizmo House' residents helped by technology Cranberry farms go high-tech to meet demand Innovation Section Front   Add Innovation headlines to your news reader: