Poker bots

Right I am in the process of making my own poker bot.
I have already finished the calculator part of the bot, so I can calculate my hand equity against known opp hand ranges.
I am currently working on Optical Character Recognition (or OCR) using publicly available OCR software, specifically GOCR.
The site I am using does NOT have a rule against bots, and when I emailed them, they replied that they do not have an anti-bot policy. Its also worth pointing out they don't state they allow bots either. But from some basic research I know the site is a safe haven.

The OCR is to record the chat box, players name, stack sizes, pot size, and bet amounts. I aim to compile stats on all the players the bot faces, from how often they fold, to how often they win etc.

My aim is not to get rich or make millions, but more to see if I can make a bot that can profit long term (1m+ hands).

Do you think bots are ethical? (I won't stop if you disagree ;-)
Do you think bots are able to make a profit long term?
Do you think bots are beatable if people know they are playing a bot?

Quote: WizardofEngland

Do you think bots are ethical? (I won't stop if you disagree ;-)

No more or less so than using a chess computer in a chess tournament.

Quote:

Do you think bots are able to make a profit long term?

A small profit at microstakes, although I don't as much think as know empirically. At medium stakes and above, where there is less fish to milk, not as of now.

Quote:

Do you think bots are beatable if people know they are playing a bot?

Bots are currently beaten by skilled players anyway, but players who are being beaten by bots now won't be able to beat them even if they have the whole algorithm and source code at their disposal.

Bots' weakness is not predictability, on the contrary, they tend to be a lot more random than people, or at least look that way to a casual observer. The issue is more related to the well-known concept of levels of thought. Poker bots have a hard time getting past Level 1; essentially, they play a perfect Level 1 game, but read only the most clear Level 2 signals, and are perfectly blind above it, even if they can throw in an occasional bluff. This lets them consistently beat Level 0 players, but they don't stand up to Level 2 ones.

My opinion is that good bots can beat low-limit games online. However, I think the best human players are much better than the best bots. I'd be okay with using a bot online as long as it wasn't specifically forbidden.

On this topic, I saw a game at the Bellagio that specifically was playing against a bot. As far as I could tell there was no rake or rules favoring the machine. I played it for about ten minutes and came out well ahead, but I'm sure that was mostly luck. The game is near the poker room and uses dollars as the denomination. As I recall, it was a $2/$4 structured game. The dealer position alternates, starting with the machine. I'm surprised there is not more buzz about it.

Quote: WizardofEngland

Do you think bots are ethical? (I won't stop if you disagree ;-)

If the poker room / web site allows it, then there's no problem. If they do not allow it, then it's not only un-ethical, but shuold be considered a criminal offense.

Quote: WizardofEngland

Do you think bots are able to make a profit long term?

Only with the right programming. But that's the same as a live player. With the right experience and poker knowledge, he can win in the long term.

Quote: WizardofEngland

Do you think bots are beatable if people know they are playing a bot?

Again, it depends upon the programming.


Calculating the odds, outs, pot equity and all that stuff is simple.

The hardest part of the program concerns bluffs. How often will the bot bluff or semi-bluff? Will it's game style change after being caught in a bluff? Will it be able to read another player's bluff?

Good luck with the project, but be warned: You're about to undertake a MAJOR project.

Quote: DJTeddyBear

Calculating the odds, outs, pot equity and all that stuff is simple.

The hardest part of the program concerns bluffs. How often will the bot bluff or semi-bluff? Will it's game style change after being caught in a bluff? Will it be able to read another player's bluff?

Good luck with the project, but be warned: You're about to undertake a MAJOR project.

About 5 years ago, I attempted a bot. It's what got me into programming in the first place. It was incredibly crude, but it worked. It made massive profit at play money (no surprise there) and just about broke even at microstakes. Which allowed me to redeem sign up bonuses in record time, which was pretty good at the time. But it was always simulating the odds against random hole cards. I now have some code that simulates against known hand ranges. Which makes all the difference, this can make the bot think like a level 2 thought process player. I think I can possibly teach it level 3, and so make a profit. I am aware of how big the job is, I have spent two weeks teaching it to read, and it still had trouble with o O and 0 but I finally cracked it this morning. It can now record every action every player makes at 4 tables simultaneously. I now need to decide how the bot uses the data, to become a formidable player.

Quote: WizardofEngland

Which makes all the difference, this can make the bot think like a level 2 thought process player. I think I can possibly teach it level 3, and so make a profit.

You'll be lucky if you get halfway into level 2. It's not OCRing the data that is hard (some bots just intercept what is sent to the client), it's using it; a semi-random bluff is not level 3 thinking. There is a number of pokerbots out there, from FL-only helloworlds to smoke and others with some finesse. None of the good ones strays noticeably from level 1 play. Level 1 is your bot's strength, where it's better than any human, anything past that is its weakness.

Bots are detectable by such things as time of response, lack of flexibility, etc.
I don't see much difference between a bot and a player but it seems the players generally think bots are evil, or atleast that an undisclosed bot is evil.
The problem of course is that bots often are several players who share information... the other players are really the same bot and often enter a table at the same time and leave at the same time.

Quote: P90

You'll be lucky if you get halfway into level 2. It's not OCRing the data that is hard (some bots just intercept what is sent to the client), it's using it; a semi-random bluff is not level 3 thinking. There is a number of pokerbots out there, from FL-only helloworlds to smoke and others with some finesse. None of the good ones strays noticeably from level 1 play. Level 1 is your bot's strength, where it's better than any human, anything past that is its weakness.

I disagree, I think level 2 is very doable. I can already get it to speculate probable starting hands for opponents, based on known hole cards (using history), position and bet size/raise size. It's not an exact science, but think I can then get it to narrow down the range by using further information on the flop turn and river.

I only really went for OCR as it is not as detectable as hooking, plus I am unable to achieve the hook I want.

As for level 3, only time will tell.

Quote: FleaStiff

Bots are detectable by such things as time of response, lack of flexibility, etc.
I don't see much difference between a bot and a player but it seems the players generally think bots are evil, or atleast that an undisclosed bot is evil.
The problem of course is that bots often are several players who share information... the other players are really the same bot and often enter a table at the same time and leave at the same time.

Time of response is easily masked. As for lack of flexibility, unless you know the constraints the bot is operating under, how would you know?

I agree, players think that bots are evil. But a bot made by me is only ever going to be as good as I play (if that). For the record, I win more at tourneys than cash.

I would never break the rules of the poker site in question, as collusion is not something I ever want to consider.

Quote: WizardofEngland

I disagree, I think level 2 is very doable. I can already get it to speculate probable starting hands for opponents, based on known hole cards (using history), position and bet size/raise size. It's not an exact science, but think I can then get it to narrow down the range by using further information on the flop turn and river.

But never near as well as a medium-skilled player. You are limited to either using predefined classifications or a weighted coefficient adjustment, while a player can often actually pin another on a hand. Of course, you have to take some level 2 considerations into account, such as increasing fold bias against tight players, but the spectrum of factors required for fully moving to that level is not well algorithmized.

One can implement algorithms that can be considered level 3, 4, 5, and why not even 6. That is not in question; the hard part is not playing at level 6, but getting better results from that than from playing at level 1. At least some poker bot makers eventually went down in complexity, because increasing alterations in basic play resulted in a decreased effectiveness. But no reason you can't try, of course.

Quote: P90

But never near as well as a medium-skilled player. You are limited to either using predefined classifications or a weighted coefficient adjustment, while a player can often actually pin another on a hand. Of course, you have to take some level 2 considerations into account, such as increasing fold bias against tight players, but the spectrum of factors required for fully moving to that level is not well algorithmized.

One can implement algorithms that can be considered level 3, 4, 5, and why not even 6. That is not in question; the hard part is not playing at level 6, but getting better results from that than from playing at level 1. At least some poker bot makers eventually went down in complexity, because increasing alterations in basic play resulted in a decreased effectiveness. But no reason you can't try, of course.

Ok, its pretty obvious you know what your talking about, but if a human can pin someone on hand, why not a bot? After-all the bot is going to know the players history flawlessly, where as the human would need to remember everything. The human brain uses logic, so can the bot, so what does the human have that the bot cannot replicate to pin someone on a hand?

For example, a raise under the gun usually signals huge strength, so a human might put the player on a big pocket pair, or at least ace-king, or a stone cold bluff. Why could I not teach the bot the same logic? If the bot ever got to see the cards in this situation it could validate (or discredit) its own assumption about that particular player.

Quote: WizardofEngland

Ok, its pretty obvious you know what your talking about, but if a human can pin someone on hand, why not a bot? After-all the bot is going to know the players history flawlessly, where as the human would need to remember everything. The human brain uses logic, so can the bot, so what does the human have that the bot cannot replicate to pin someone on a hand?

"What would I do" thinking. No human player is all tight, or all loose, or all level 1, 2 or 3, everyone varies his play. A human can put himself into his opponent's shoes, because they both think the same way, but a program doesn't.

Quote: WizardofEngland

For example, a raise under the gun usually signals huge strength, so a human might put the player on a big pocket pair, or at least ace-king, or a stone cold bluff. Why could I not teach the bot the same logic? If the bot ever got to see the cards in this situation it could validate (or discredit) its own assumption about that particular player.

Of course, you can teach the bot that. It's just that sometimes a human will not have that hand, either because he isn't considering position, or sends the signal intentionally. And then, it's difficult to combine the first cue with further cues, I just can't think of a way to algorithmize that, while recalling when I played just like that myself and comparing it is pretty easy. The issue is the cost of mistakes being pretty high, so it's sometimes better to ignore mixed signals and just level down to ABC strategy, even for a human player. For a bot, such an option is much more often favorable, because it's inherently weaker at complex tactics, but very strong at playing by the book - check the outs, subtract false outs, play the implied odds - even good players can't do it as well as a computer.

Quote: Wizard

My opinion is that good bots can beat low-limit games online. However, I think the best human players are much better than the best bots. I'd be okay with using a bot online as long as it wasn't specifically forbidden.

On this topic, I saw a game at the Bellagio that specifically was playing against a bot. As far as I could tell there was no rake or rules favoring the machine. I played it for about ten minutes and came out well ahead, but I'm sure that was mostly luck. The game is near the poker room and uses dollars as the denomination. As I recall, it was a $2/$4 structured game. The dealer position alternates, starting with the machine. I'm surprised there is not more buzz about it.

The Bellagio game got a lot of attention at the TwoPlusTwo forums some time ago.

The game is fair per Vegas regulations.

The reason this is possible is that heads up limit holdem has been theoretically 'solved'. The unexploitable strategy for a heads-up, capped, limit holdem game is just barely within the limits of what can be reasonably calculated with computers, from a computational complexity perspective. This is according to Bill Chen and Jerrod Ankenman, authors of "The Mathematics of Poker".

Strangely, the information available from various sources (mostly from forums and web searches) suggests that a brute force computational approach was not used to develope the strategy for this game. Instead, it was revealed that the strategy was arrived at by an addaptive or evolutionary programming set up, where the program was designed to pick a starting strategy and slowly refine it over many thousands of hands of 'practice'. My guess is that the final strategy the program came up with is probably fairly close to the unexploitable strategy to maintain an edge over the vast majority of players. I am going from memory with this part, but this is what I recall reading on the TwoPlusTwo forums, which had some reasonably reliable information in it.

Rest assured that NL Holdem' isn't really very close to being 'solved'. The complexity of the NL calculations would be orders of magnitude greater than the heads up capped limit holdem game.

As far as bots go, I think that the iterative or adaptive approach is probably the best way to go. I also have no doubt that there are many people making very small profits from poker bots. Free-roll tournements and low stakes games seem like likely targets.

Anyone intereseted in the subject should read the information from the University of Alberta Computer Poker Research Group: