Math question for backgammon !

With this subject, I would like to try and quantify skill in backgammon. For consideration, take two good players but one of them is just 1% better (consider this as fact and exact figure) than the other. So, statistically, out of 1000 match games player A should win 505 and player B should win 495.

I have a double question: 1) what is the minimum amount of match games that player A should play against player B to be 90% sure to come out as a winner on overall ?
2) what is the minimum amount of match games that player A should play against player B to be 99% sure to come out as a winner on overall ?

Story behind this is that many backgammon players (including me) seem to have no idea what "long run" really means. It is just generally agreed that the better player will overcome the luck factor and will win in the long run. OK, but when the level is that close ?

I would see this 1% like a biased coin flip but really don't know the answers.

Quote: PlayHunter

I have a double question: 1) what is the minimum amount of match games that player A should play against player B to be 90% sure to come out as a winner on overall ?

about 16,600

Quote: PlayHunter

2) what is the minimum amount of match games that player A should play against player B to be 99% sure to come out as a winner on overall ?

about 55,000

a guess, maybe 1 out of 100,000 really knows what 'the long run' actually means

7craps, thanks ! But how you arrived to that numbers ? Which formula did you used ?

So to be 99% sure that I am ahead of a player which is just 1% weaker than me would take 8 hours of play, everyday, for about 9.5 years (considering a match = 30 min).

PS: And I guess, that if there would be 2% difference in skill instead of the 1%, then the figures would be halved ?

Quote: PlayHunter

7craps, thanks ! But how you arrived to that numbers ? Which formula did you used ?

I used the binomial probability formula. pari/gp calculator works just fine as does Excel (I think Excel uses the normal distribution over 1,000 trials. ok)

Quote: PlayHunter

PS: And I guess, that if there would be 2% difference in skill instead of the 1%, then the figures would be halved ?

guessing at probability often times results in a wrong guess (close to 100% of the time)
about 4,200 match games at 90%
24,000 at 99%

Now that we established how many match games would be needed to wipe out the luck factor from the calculation, I do have another question on the same subject.

I would like to find out how correct I was to use this 1% difference in skill to start the calculation with ?

This 1% difference in skill was statistically determined from a database with 10000 match games to 5 point (the player who reaches 5 points first, wins the match) played. All match games were analized with a backgammon software which can calculate the error rate of each player. (100 match games to 5 points are super enough for the software to accurately determine the error rate of each player. Even 70 match games to 5 points are quite enough for the program to determine how good a player is.)

After the 10000 match games were played, the result was that the better player won by 5050 against the weaker player. So from there the 1% figure I used.

But I'm afraid that this 10000 match games database might not be really big enough for the 1% figure to be really relevant ? (By really relevant I mean, +/- 10% accuracy.) If that's the case indeed, then how many match games in the database would be enough to make the result really relevant (by +/-10% accuracy) ?

Let me see if I have the question right. Player A has a 49.5% chance of winning a single game. I show that he will win a "first to 5 wins" tournament with probability 0.487697. If the question what is the least number of tournaments that need to be played for player B to have at least a 90% chance of winning more? I show the answer to that question to be 2,661. For a 99% chance of player B winning more, I show the least number of tournaments should be 8,853.

One can solve this with either the Gaussian curve approximation or binomial distribution. It is so easy to do either in Excel. When I was in college we had to use statistics tables and trial and error. My statistics book is falling apart from going to and from the standard normal table thousands of times.

Quote: Wizard

Let me see if I have the question right. Player A has a 49.5% chance of winning a single game. I show that he will win a "first to 5 wins" tournament with probability 0.487697. If the question what is the least number of tournaments that need to be played for player B to have at least a 90% chance of winning more? I show the answer to that question to be 2,661. For a 99% chance of player B winning more, I show the least number of tournaments should be 8,853.

One can solve this with either the Gaussian curve approximation or binomial distribution. It is so easy to do either in Excel. When I was in college we had to use statistics tables and trial and error. My statistics book is falling apart from going to and from the standard normal table thousands of times.

Thank You, Wizard !

I don't think you got the question correctly. The better player has 50.5% chances to win a match game (not a single game). A match game is composed of multiple single games. A single game can be won with 1, 2, or 3 poins. A "match to 5" means that the player first to reach 5 points, wins the match. The 1% advantage was per match to 5 (let's call it a battle) and not per single game.

There was a database of 10k battles (the format of each battle was: the first player to score 5 points in a battle, wins the battle). The better player won 5050 battles and lost 4950 battles . From that, I concluded that the better player was 1% better than the other.

My question, from my last post meant to be: was this 10k battles database large enough for the result to be relevant ? I mean, if the same players will play another 10k battles, will the results be very similar again ? Will the same player win again somwhere between 5040 to 5060 battles or there may be a much different result ?

PS: Or shall I better ask, what chance is that the 5050 wins out of 10k battles were correctly reprezenting that the better player is indeed only 1% better and not say 1.5% better but been unlucky or he is only 0.5% better skilled but been pretty lucky during these 10k battles and actually won quite more than he should ?

I think with being able to win 1, 2, or 3 points in a game, the math is getting rather messy. I'll pass the ball to 7craps.

Quote: Wizard

I think with being able to win 1, 2, or 3 points in a game, the math is getting rather messy. I'll pass the ball to 7craps.

I think you misunderstand. The points themselves appear to be irrelevant. Player A has a 50.5% chance of winning any match, and Player B has a 49.5% chance of winning that same match.
The questions are:
(a) What is the smallest number N such that the probability of Player A having won more matches than Player B after exactly N matches is 0.9?
(b) What is the smallest number N such that the probability of Player A having won more matches than Player B after exactly N matches is 0.99?

Quote: ThatDonGuy

I think you misunderstand. The points themselves appear to be irrelevant. Player A has a 50.5% chance of winning any match, and Player B has a 49.5% chance of winning that same match.
The questions are:
(a) What is the smallest number N such that the probability of Player A having won more matches than Player B after exactly N matches is 0.9?
(b) What is the smallest number N such that the probability of Player A having won more matches than Player B after exactly N matches is 0.99?

ThatDonGuy, thanks but I think 7craps answered that two questions.

My next problem was to determine if the win ratio was made on a big enough number of events (10000 battles) to pretty accurately determine that the win ratio should indeed be 50.5% for the better player and not some other figure.

So the question now is: what chance is that the 5050 wins out of 10k battles were correctly reprezenting that the better player is indeed only 1% better and not say 1.5% better but been unlucky or he is only 0.5% better skilled but been pretty lucky during these 10k battles and actually won quite more than he should ?

Quote: PlayHunter

So the question now is: what chance is that the 5050 wins out of 10k battles were correctly reprezenting that the better player is indeed only 1% better and not say 1.5% better but been unlucky or he is only 0.5% better skilled but been pretty lucky during these 10k battles and actually won quite more than he should ?

One (1) binomial standard deviation of the sample size of 10,000 trials for 0.505 = 0.499974999 (.5)
The standard deviation of p(.505) = .5%

The 6 sigma would be 51.9999% to 49.0001%
a very large range when one really looks at it (very large sample error due to the small sample size)

Quote: 7craps

One (1) binomial standard deviation of the sample size of 10,000 trials for 0.505 = 0.499974999 (.5)
The standard deviation of p(.505) = .5%

The 6 sigma would be 51.9999% to 49.0001%
a very large range when one really looks at it (very large sample error due to the small sample size)

Yep, I was afraid that the 10,000 sample might be too slim !

If so, how many samples would be needed to (pretty) accurately determine the difference of skill between the two ?

Quote: PlayHunter

If so, how many samples would be needed to (pretty) accurately determine the difference of skill between the two ?

depends on what confidence level one wants and the error.
at 99%, 1,000,000 trials gives a z_error = 0.00128785
p_est: 0.505
error+: 0.50628785
error-: 0.50371215
skill: (0.012575701 to 0.007424299)
this is statistics 101 (z_score/error)^2*(p*(1-p))

many in the poker world think 10,000 hands played is the 'long run'
just to find out later it it just the 'tip of the iceberg'

Quote: 7craps

...many in the poker world think 10,000 hands played is the 'long run'
just to find out later it it just the 'tip of the iceberg'

Love how you put that. I'm all clear now, TYVM !