How Many Trials?

Let's say that you're betting on the outcome of a fair coin flip. If it's heads, you win a dollar, and if it's tails you lose a dollar. You have to play 99 times. The Standard Deviation for this scenario is about $9.95. My question is: How many sessions of 99 coin flips ( each 99 coin flip session counting as 1 ) could you expect to perform before you get a result that is outside of 5 standard deviations ( win or lose >= $50 )? I know there's a way to guestimate this but I haven't wrapped my head around the process.

I only ask because I saw some news recently about some scientific result at Fermilab that was outside of three standard deviations but Scientists are only convinced of a result if that result is outside of five standard deviations.

I am going to change your question and imagine you are asking how you get a result that is [say] likely to happen say once in 10,000 trials, if that is where 5 standard deviations put you. A quick answer is always that there is roughly something like a 50-50 chance of something happening at least once in such a case with 10,000 trials, or X trials if chances are 1/x. To do the math, you work out there is 9999 chances out of 10,000 that it won't happen each trial, and get your number. In this case I get 0.36786104643293, or about one chance in 3 that it would never happen and thus about 2 out of 3 chances you will get at least one result beyond 5 st. dev. in 10,000 trials. Bear in mind I am guessing as to where 5 standard deviations put you.

The next part is a puzzle to me. Scientists will have a model that predicts a result and might view a result outside 3 standard deviations as "no proof" the model was wrong? much as we might say someone who had a bad day gambling should get over it. If however the gambler can prove he played craps for 6 hours and only won a couple of hands, we can agree he had extremely bad luck and may have been cheated with a crooked game. Scientists should view a result outside 5 st. dev's as meaning they had a bad model? I'm not sure otherwise what 5 standard deviations would mean to a scientific experiment. Perhaps someone can help.

Nevermind, I found it. Thanks Wikipedia.

zσ	Percentage within CI	Percentage outside CI	Fraction outside CI
0.674σ	50%	50%	1 / 2
1σ	68.2689492%	31.7310508%	1 / 3.1514872
1.645σ	90%	10%	1 / 10
1.960σ	95%	5%	1 / 20
2σ	95.4499736%	4.5500264%	1 / 21.977895
2.576σ	99%	1%	1 / 100
3σ	99.7300204%	0.2699796%	1 / 370.398
3.2906σ	99.9%	0.1%	1 / 1000
4σ	99.993666%	0.006334%	1 / 15,787
5σ	99.9999426697%	0.0000573303%	1 / 1,744,278
6σ	99.9999998027%	0.0000001973%	1 / 506,800,000
7σ	99.9999999997440%	0.0000000002560%	1 / 390,700,000,000

Quote: s2dbaker

Nevermind, I found it. Thanks Wikipedia.

Note that I take a verbal approach to math. I know now that this is a sign a person will not do well in math.

edit: looks like that chart might show that 5 st. deviations is 1 / 1,744,278 , approaching 1 in 2 million. So I did get something out of that maybe! [g].

I was formatting my answer as you were posting yours. I didn't see it until now. But i'll have a good look at it when I get back home from work.

Quote: s2dbaker

I was formatting my answer as you were posting yours. I didn't see it until now. But i'll have a good look at it when I get back home from work.

there is a 50% chance of getting a result that is in excess of 5 standard deviations after 1,208,837 trials.
Keeping to the 5 standard deviations, you would have to run the trial 17,035,802 times to make scientists happy.

Here's a fun one. If you flip a coin and force the win/loss to one unit then the standard deviation will equal the square root of the number of trials. 100 coin flips and the std dev = 10, 289 coin flips and the std dev = 17. If your "coin" is not a +1 and -1 but instead a random number generator, your outcomes will have to fall between -SQRT(3) and +SQRT(3) to get a std dev that's equal to the square root of the number of trials.

I don't know why the square root of 3 is so special.

Quote: s2dbaker

Keeping to the 5 standard deviations, you would have to run the trial 17,035,802 times to make scientists happy.

For the second part of this question you originally posed, I wonder to what standard this 5 sd's really applies? Certainly experiments are done all the time with just a few trials. For an example, Pasteur famously sealed off some glass flasks to prove to his satisfaction that a sterilized medium will remain sterile permanently if sealed off from the outside. Did his colleagues start demanding he do millions of flasks or the results could not be accepted?

Quote: odiousgambit

For the second part of this question you originally posed, I wonder to what standard this 5 sd's really applies? Certainly experiments are done all the time with just a few trials. For an example, Pasteur famously sealed off some glass flasks to prove to his satisfaction that a sterilized medium will remain sterile permanently if sealed off from the outside. Did his colleagues start demanding he do millions of flasks or the results could not be accepted?

your approach may be flawed. There should be a comparison between flasks that are not sealed and ones that are. If 100 flasks are left unsealed and 50 become contaminated but 98 of the sealed ones remain uncontaminated, then that may qualify as a 5 stddev event.

Quote: s2dbaker

your approach may be flawed. There should be a comparison between flasks that are not sealed and ones that are. If 100 flasks are left unsealed and 50 become contaminated but 98 of the sealed ones remain uncontaminated, then that may qualify as a 5 stddev event.

I was thinking about that. Pasteur's Hypothesis was that it requires outside contamination to spoil the medium, not that there is a percentage of time the medium stays unspoiled. So you can't say he was testing that 10 billion times out of 10 billion and one it remains uncontaminated; the hypothesis is "never" and no number of trials makes sense with that. If *any* sealed get spoiled it disproves the hypothesis completely. Even then it has to be determined that the medium was truly sterilized to begin with, with some bacteria that requires quite a bit.

But I am still wondering what requires 5 standard dev's. Certainly the Hypothesis has to be that something will happen 1 in X times, then that is tested. My example wasnt good, you are right.

I aslo pulled those number from whence the sun don't shine. I have no idea what qualifies as scientific evidence.