Fix my (Java) blackjack simulator?

I need to hire someone to fix my Java-based blackjack simulator. All the starting hands show the proper return except the split hands, so my split code is faulty somehow. Actually, the return for paired hands (like 5-5) is correct as long as the strategy is not to split. For any hand that's actually split, my result is wrong. But the result isn't off by much, which makes the error really hard to find by examining the output of lots of hands played.

For example, according to the Wizard's EV table, 7,7 vs. 10 should return -0.48991. My sim returns -0.489922 after 1.5 billion hands, so the difference of -0.00001 is essentially correct.

But for 9,9 vs. 3, the return should be 0.239235 while my sim gives 0.239508 after 10 billion rounds, which is off by 0.0027, which is clearly wrong.

I've looked at all the obvious things: making sure I cleared out arrays from the previous round, making sure we can resplit up to 4 hands, making sure doubles work properly, but clearly I'm missing something.

Who's got Java skills, and what's a fair price to find the error in my code?

I've got Java skillz, but I'm afraid my math skills aren't too spectacular. I guess as long as I can compare the output of the simulator to the EV table, that won't necessarily be a problem.

If you'd really like to hire some freelance work, I can give you my resume.

The fair price part is trickier... for something like this, I'd probably do something like an hourly rate I guess.

I can take a look ... I won't charge you for this, and will either find the error, or tell you that it's too much work to do for free, and give up.
If you like this conditions, PM me your code (and, preferably a test case demonstrating the problem).

Also, I noticed this comment in Wizard's page:

Quote:

The values in this table were created using a recursive combinatorial program that considered the exact hand composition at every step. For splitting values the program only considered how many hands the player had already split, but not the other cards drawn in those hands.

Could this be the cause of the discrepancy? Is your sim using composition-dependent strategy? And, more importantly, is it returning played cards into the shoe after evaluating each of the split hands?

Thanks for the replies. I'm giving weaselman a crack at it first. (I'm familiar with him because he was helpful in my thread about overhauling the betting system challenge.) If that doesn't pan out, then it'll be time for me to select another Java programmer.

Quote: MichaelBluejay

Thanks for the replies. I'm giving weaselman a crack at it first. (I'm familiar with him because he was helpful in my thread about overhauling the betting system challenge.) If that doesn't pan out, then it'll be time for me to select another Java programmer.

Makes sense. I'm here if you need me.

Yes, but
http://www.bjmath.com/bjmath/ev/2ds17.htm
gives 0.23964 for 9,9 v 3. Exactly what information are you using to evaluate the EV of a resplit?

I'm not evaluating the EV of a resplit. My understanding is that the Wizard calculates the EV only on the original wager, and that's what I do, too.

I think it is worth verifying if we're doing it the same way. Let's look at the case of 9,9 vs. 3. Let exp(x) be the splitting expectation for one hand with cards x removed. Let pr(x) be the probability the player re-splits to x hands. Then my overall expected value would be:

pr(2)*2*exp(9,9,3) + pr(3)*3*exp(9,9,9,3) + pr(4)*4*exp(9,9,9,9,3).

Also, in calculating exp(9,9,3), for example, the first card drawn after each 9 can't be another 9, because we declared the player only got to 2 hands.

This get all the more complicated when the dealer has a 10 or ace up, but interestingly we agree in that situation.

If weaselman doesn't find the problem you might consult with Cindy Liu. She does it the same way I do. In fact I had to consult with her because my results didn't agree with hers at first.

Thanks for the help. I'm not calculating the EV at all, I'm finding it through simulation. I allow resplits to four hands (except aces), because that's what the rules the top of the EV table on WoO.com say. If I allow only one resplit then the EV reported by the sim dives and is even farther off.

Quote: MichaelBluejay

Thanks for the help. I'm not calculating the EV at all, I'm finding it through simulation.

Well, yes, but then you are comparing it to the EV, calculated by Wizard, and the way he did it is an approximation, that does not exactly match your real-life simulation. Consider his ex(9,9,9,9,3) for example. The same expectation is used for all four hands, but in reality there should be four different values, because after the first hand is played, there are fewer cards left in the deck.
I believe, that is the reason for the discrepancy, but do not have proof yet. I have modified your program to return played cards (except for the first one) back to the shoe after each of the split hands is played. That got me a different result, that still does not match Wizard's number though, but now is lower than the target, not higher. I believe, this is because you rearrange the cards to put the ace first, so the card that doesn't get returned to the deck with my modification is not always the same that was dealt initially.

I have not had the time to fix this last problem yet, but did look through the code fairly thoroughly and don't see any logical errors there, so I am fairly confident that this is indeed the cause of the discrepancy.

And one more thing. You are off by an order of magnitude calculating the difference between your result and Wizard's. It is not 0.0027, but rather 0.00027, or about 0.11% of the actual value, which does not seem too bad.

Weaselman, thanks for your help.

I agree that it's possible the problem isn't with the split code. That just seemed like the smoking gun, since my split numbers don't match the Wizard's, but the non-split numbers are dead-on.

So let's back up and I'll explain why I thought there was a problem in the first place. If I reshuffle every hand, the return ought to be -0.1923%, according to the Wizard's appendix 15. My sim returns -0.2365%, so I thought I was off. If I use a cut card, then the return ought to be -0.25532% according to the Wizard's house edge calculator, but my return is -0.2382%, so again I'm off.

Thinking that I was off, I examined the results of the specific starting hands, and discovered that my non-split hands are dead-on, while the split hands are consistently off.

I'm at a loss as to where to go from here.

Weaselman, I'm not sure I can agree that 10 billion rounds isn't enough. I find extremely little variation after just a few billion rounds. I let it go to 10 billion for overkill, but it does seem like overkill. I can't explain why your version got so far away, but my experience has been different. Here are the results of my last test:

10M -0.2286%
15M -0.231%
20M -0.2346%
100M -0.2234%
500M -0.2387%
1000M -0.2384%
2000M -0.2354%
3000M -0.2372%
4000M -0.2373%
5000M -0.2384%
6000M -0.2381%
7000M -0.2380%
8000M -0.2384%
9000M -0.2384%
10000M -0.2382%

Quote:

So let's back up and I'll explain why I thought there was a problem in the first place. If I reshuffle every hand, the return ought to be -0.1923%, according to the Wizard's appendix 15. My sim returns -0.2365%

That still does not mean that the problem has to be with the code. If Wizard's numbers for individual hands are a little off because of the approximating, then the overall return would be off for the same reason.

Interestingly, your overall number is lower than Wizard's while the 9,9,3 one is higher for you (and also, as I said before, the 9,9,3 difference is really quite small to begin with), meaning that either there must be another hand with a higher deviation into the negative side, or you are calculating the overall wrong. Can you compare the results for all hands and see what the largest difference is?
Or else, you may be using a slightly different strategy. Are you absolutely positive all your plays, including the composition-dependent exceptions are the same as what Wizard used?

One more thing is that your sim never surrenders. Do you know that that's the case in the Wizard's table? If not, it might very well account for the difference.

Quote:

I can't explain why your version got so far away, but my experience has been different. Here are the results of my last test:

Well, I was running it specifically for the 9,9,3 combination when it seemed that 10B wasn't enough. I am thinking that splits are particularly sensitive.
I actually removed that comment later, because I myself find it so hard to believe that a number as huge as 10B can be too small for anything, but I saw what I saw. BTW, the numbers did look fairly stable to me to me too after a few billion rounds (more or less, same as in your output), but when I let it run for a long time, and came back, they evidently have slowly drifted quite far away from where they appeared to be getting stabilized at first.
I'll run it again today ...

I ran the sim for most splits, but gave up when it became clear that the splits, and only the splits were off. Here's how much I'm off by vs. the Wizard's numbers. Positive numbers mean that my sim shows a lower house edge than the Wizard, negative numbers mean I show a lower return than the Wiz:

9,9 v 9: 0.015%
9,9 v 8: 0.005%
9,9 v 7: 0.007%
9,9 v 5: 0.013%
9,9 v 4: 0.018%
9,9 v 3: 0.027%
9,9 v 2:-0.056%
8,8 v10: 0.028%
8,8 v 9: 0.075%
8,8 v 8: 0.026%
8,8 v 7:-0.005%
8,8 v 6: 0.002%
8,8 v 5: 0.017%
8,8 v 4: 0.017%
8,8 v 3:-0.175%
8,8 v 2:-0.056%
7,7 v 8: 0.044%
7,7 v 7: 0.014%
7,7 v 6: -0.008%
7,7 v 5: -0.003%
7,7 v 4: -0.102%
7,7 v 3: -0.183%
7,7 v 2: -0.171%
6,6 v 7: 0.002%
6,6 v 6: -0.016%
6,6 v 5: -0.137%
6,6 v 4: -0.125%
6,6 v 3: -0.102%
6,6 v 2: -0.243%

I don't think the Wizard assumed surrender in the two-deck game. If he did he would have said so in the notes.

I'm pretty sure all my plays, including composition-dependent ones, are correct, because all the non-split hands return perfect results.

About your returns slowly drifting after huge sessions, that could be due to some kind of overflow, memory filling up somewhere or something, or maybe patterns in the RNG with such a large period. I'm still pretty confident that 10B hands is enough that the difference between actual and expected should be less than 0.01%.

This is a stumper, huh? I don't feel quite as bad now, since you haven't found the problem yet either.

Quote: MichaelBluejay

9,9 v 3: 0.027%

I think, expressing absolute difference in percent notation is kinda misleading.
0.00027 is really only 0.114% of the actual value (0.239235), which isn't as bad as it looks.

I added up the differences you quoted weighed by their respective probabilities and got -0.000004091 - way too small to explain the difference in the overall.
(you may want to double check my math though ... I calculated the weight as 8/104 * 7/103 * P/102, where P = 6 if the upcard matches the split card value, 32 if the upcard is 10, and 8 otherwise).

Quote:

About your returns slowly drifting after huge sessions, that could be due to some kind of overflow, memory filling up somewhere or something, or maybe patterns in the RNG with such a large period.

Basically, you are saying that there is such a thing as "too many experiments". If that's the case though, how do we know that 10B isn't too many already? :) Maybe, your value was close to Wizard's number somewhere around, say, 6 billion, and the drifted away from it for the same reasons you mention?

Well, the thing is, the non-split hands are correct to within 0.001% (or -0.00001, if you prefer). But my 6,6 v 2 figure (for example) is off by *243 times* that amount!

On the drifting thing, I do remember once when the output of the number of rounds starting going *backwords*, and then it went *negative*, and then it rebounded and went positive, then the whole thing repeated. I don't claim to understand all this well. And you're right that it's possible that I've already gone far enough to encounter some kind of overflow or repeat problem.

Maybe I should just consider the sim good enough? It is pretty close.

In that case, let's first look at your RNG. Many (most) built-in RNGs are not good for Monte Carlo simulations. What are you using?

I'm using whatever's built-in with Java. But again, all the non-split hands are fine. It's only the splits that are off.

Quote: MichaelBluejay

On the drifting thing, I do remember once when the output of the number of rounds starting going *backwords*, and then it went *negative*, and then it rebounded and went positive, then the whole thing repeated. I don't claim to understand all this well.

Yeah, that's normal. The highest (32nd) bit in the java representation of an integer is used for the sign. So, when the number grows beyond 2^31-1, instead of becoming 2^32, it turns to a huge negative number. If you keep incrementing it, it'll eventually reach zero, and cycle around. That's why I said earlier that I did not know how many rounds the program actually run. (With longs, it is the same story, except replace 32 with 64).
But that's just the counter, it should not have any effect on the results.

Quote:

And you're right that it's possible that I've already gone far enough to encounter some kind of overflow or repeat problem.

The only thing I can imagine remotely have to do with anything is the RNG pattern. I went ahead and changed your program to reinitialize the RNG after every 100000 numbers. Not surprisingly, it did not make any difference.
I have also fixed my earlier change to correctly return played cards to the deck after each split hand, but the result it produces is still significantly lower than the Wizard's.

Quote:

Maybe I should just consider the sim good enough? It is pretty close.

Personally, I think, it is. But it sure is intriguing to figure out the root cause of the discrepancy.

Okay, mystery solved. Weaselman figured it out, but neither of us realized it at first.

Weaselman notified me by PM of an error he found where I was splitting aces, but we both didn't think that was the problem because that wouldn't account for the other splits being wrong, and that's where we thought the problem was. But remember that the whole reason I was looking at individual hands in the first place was that my total-game return was off? After I fixed the error Weaselman found and ran the whole-game sim again, I get the correct result.

In fact, I tried to run the whole-game sim again earlier after I fixed the problem he found, but when I did so I forgot to set the sim back to resplit to four hands. (I'd previously changed it to resplit to 2 hands to see how that affected the numbers for the individual split hands.) So when I ran the whole-game sim the first time after fixing the error, I was splitting to only two hands, and so my whole-game result was still wrong.

Now that I fixed the ace-splitting error and I'm properly resplitting up to four hands, the sim returns the proper result:

Reshuffle each hand: Expected = -0.1923%, My sim = -0.1918% (after 10 billion hands)
With a cut card: Expected = -0.25532%, My sim = -0.2513% (after 6 billion hands)

So what does this mean about the split hands being off? They're probably not really off. The Wizard is calculating the figures and I'm simulating them, my results are actually probably good.

So Weaselman wins again. This is the second time he's solved a problem for me. Weaselman, how can I reward you? Feel free to PM me if you prefer.

Quote:

Weaselman notified me by PM of an error he found where I was splitting aces, but we both didn't think that was the problem because that wouldn't account for the other splits being wrong, and that's where we thought the problem was.

Wow. An extreme case of asking a wrong question :)

And here I am writing my own analyzer for the splits to double check Wizard's results because I can't find anything else wrong with the sim :)

Quote: MichaelBluejay

Weaselman, how can I reward you?

Your praise is my reward :)

Don't worry about it, I just like riddles, so it's my pleasure to help.

I think Mark Twain noticed once that "Intellectual work is misnamed; it is a pleasure, a dissipation, and is its own highest reward".
He also said:
"There are wealthy gentlemen in England who drive four-horse passenger-coaches twenty or thirty miles on a daily line, in the summer, because the privilege costs them considerable money; but if they were offered wages for the service, that would turn it into work and then they would resign."
Which sounds exactly like a reason why I refused to get paid for this. If there is one thing I don't need, it's more work :)