Variance and standard deviation, help me out in understanding them

Could somebody explain these to me? I looked up variance on Wikipedia, but it's so freakin complex. I read that variance is the square of the standard deviation I think? Why have both then? I understand how to read the numbers, such as a variance of 20 on video poker is very low, and 80 is very high. I know high variance means the swings will be bigger. I understand that, but what exactly do the numbers mean exactly?

Oh, and how would n-play video poker make the variance go up? (See link below.) Shouldn't it make it go down? I could swear I remember reading that n-play video poker will smooth out the ups and downs, which makes total sense. This is why I need the actual number explained to me.

http://wizardofodds.com/videopoker/appendix3.html

Something about a standard deviation being either negative or positive, but if you square it you will always be dealing with a positive number. I guess you can tell I really don't know the answer as to why there are both terms.

Both variance and standard deviation are a measure of how broad distributed you can expect random results.

Mathematicians love variance, which is the average squared distance from the mean. Variance provides easy properties for calculation of random results (most important you can sum up variances from different bets) , however - since it is a squared property (in terms of units) it is not easy to imagine.

Standard deviation, as the root of the variance, has the same unit as expectation value, and hence both standard deviation and expectation value are comparable. This is nice for calculating probabilities (i.e. you have a 68% chance that your results are within 1 standard deviation around your expectation value if you have a large number of plays). However, from the mathematical view standard deviation is quite messy, and not much useful for any serious calculations.

Most people confuse standard deviation and variance, and use the term "variance" for any kind of randomness. In common conversations this is not a big problem, when variance is low so is standard deviation (and vice versa). But if you are up to actual numbers, you need to understand.

The thing about multi-play video poker and variance is a different story. You need to introduce the concept of correlation.
Different rounds of video poker are fully independent, and as independent rounds their variance simply add up. (the standard deviation doesn't add up, as it is a square root). If you were to play 100 rounds of video poker, you would get 100 times the variance of a single round.

Now, what happens if you play a 100-hand video poker machine ? Now each hand is not independent, as you select your cards for all 100 cards simultaneous. Hence all 100 hands are correlated, which INCREASE variance for that 100-hand round. However, since you can play 100-hand video poker at a lot less stake per hand, one round of 100-hand VP at betsize 1/100 unit each hand has LESS variance than one round of a single hand VP at betsize of 1 unit.
The expectation value however is the same (as the stake is the same each round), but the variance is greatly reduced.

I don't think I'll ever understand beyond simply knowing what values are high and what are low, like knowing to stay away from something with a variance of 80 if you have a small bankroll. Is a variance of 80 the same across everything, or is it relative to something?

And I just want to know what the actual number means. 5 "deviations", or 45 "variance" OK, what is a deviation/variance? To go outside the norm, etc., I know, but 1 standard deviation is what? Within 68% of the average or something? OK well where did that 68% come from? Frustrating.

I should probably just use Google on this, but give me an example. Like a variance of 30 (or SD of 5 or something), in terms of flipping a coin, would mean...

Quote: SilentBob420BMFJ

I should probably just use Google on this, but give me an example. Like a variance of 30 (or SD of 5 or something), in terms of flipping a coin, would mean...

If examples help then you can ponder this WoO webpage.

But don't over think this. For example recently I came to realize that the statement in that webpage that says "The standard deviation of the final result over [so many] bets is the product of the standard deviation for one bet (see table) and the square root of the number of initial bets made in the session" outlines something that is actually quite easy for anyone to calculate. The word "product" indeed means multiplying these items... I was far enough removed from my days of taking math to have forgotten that at first.

What you wind up with is something that has some meaning. OK, one standard deviation should have been this much won or lost, I can compare that to what happened or what I expect to happen.

Just 2 cents from somebody who struggles with this too.

Variance of 80 is relative. There is another measure called coefficient of variance, which can be compared across games (although it isn't used often)

The 68% is an area under a pdf curve for the normal distribution.

There is a difference between the variance (or standard deviation) of a distribution, and an event being a specific # of standard deviations away from the average for that distribution.

If video poker has a variance of 80, it means that the sum of the squares of each (result - average) is 80. But when you actually play some hands of video poker, you will have a specific outcome (like, down 20 units). You can the calculate how likely it was that you got the specific outcome in terms of deviations from the norm.

so, variance of 30 for coin flipping doesn't mean anything, until you start talking about flipping a coin a certain number of times. If you flip a coin 22 times, and it is heads all 22 times, that event is 5 standard deviations away from the norm.

Quote: dwheatley

Variance of 80 is relative. There is another measure called coefficient of variance, which can be compared across games (although it isn't used often)

The 68% is an area under a pdf curve for the normal distribution.

There is a difference between the variance (or standard deviation) of a distribution, and an event being a specific # of standard deviations away from the average for that distribution.

If video poker has a variance of 80, it means that the sum of the squares of each (result - average) is 80. But when you actually play some hands of video poker, you will have a specific outcome (like, down 20 units). You can the calculate how likely it was that you got the specific outcome in terms of deviations from the norm.

so, variance of 30 for coin flipping doesn't mean anything, until you start talking about flipping a coin a certain number of times. If you flip a coin 22 times, and it is heads all 22 times, that event is 5 standard deviations away from the norm.

So, are you saying that variance in video poker is based on a certain number of hands? How many? Kinda like how mph, mpg, etc. are based on going a certain distance/amount.

And if that's not it, and what it means is that with SD of 4, it's saying at any given time, you shouldn't be more than 4 SD away, well then I have a question about that. With what % chance are they saying you will be within a SD of 4?

"This statistic is commonly used to calculate the probability that the end result of a session of a defined number of bets will be within certain bounds."

What number of bets and what bounds? For instance, for JOB VP it's 19 variance, which would make it a SD of 4.36. OK, so explain what that's saying. And I'm confused, because SD is a %, so why not just say the %? If 1 SD is 68%, why not just say you have a 68% chance? All this is confusing as hell.

Please forgive me if this starts out not being directly gambling, so here goes.

Suppose someone is making pencils, filling coke bottles, cans of food, whatever. It is almost impossible to make a pencil exactly 6" long. Typically there will be an average and, due to manufacturing, different length pencils will actually be produced. Most natural things approximate to the normal curve/distribution.

If we wanted to ensure that a certain percentage of pencils were 6" or longer (99% 95% or whatever), we would have to set the average length to be manufactured at more than 6". On the other hand, we might want to lower the variance (i.e. better quality production) and reduce the average error. The SD gives a measure of how far out we usually are. So for instance if it was .1", we might aim to produce pencils of average 6.2", so usually they would be 6.0" to 6.4". However if we were much better at manufacturing, we might be able to aim towards an average of 6.04".

Back to gambling. The usual question is each time I make a bet of (say) £1 what will happen. The average tells us that in the long term we'll get back (say) 98p per £ (or c per $ if you prefer). Obviously, normally, we'll either win multiples of £1 or nothing, and after (say) making 1000 bets we'll have won back something (or possibly nothing!) The SD gives an indication of how near £980 we're likely to be - like the pencils how far out we usually are. If the SD is large then we might make a lot of money, or have lost a lot. If the SD is low (such as even money roulette or punto banco bets) it is likely we'll be (say) between 900 and 1100 - so we would tend not to lose a lot nor win a lot - it's less "risky".

You may have heard risk associated with investments - essentially they are bets (albeit with positive expectations). A (good!) bank pays the same, called interest - little risk - no SD you know what you'll get back at end of year (X + Y%). A Dow Jones share has some risk - medium SD. A minor oil drilling company, or similar, might have major risk - the upsides are good - but the risk of losing everything is quite high! As they say you pays your money and takes your choice.

PS The reason for the 68% is that is the chance of being within 1SD.
In our exaggerated example 68% of pencils will be 6.1" to 6.3", 95% will be 6.0" to 6.4" (2 SDs), 99% will be 5.9" to 6.5" (3 SDs) etc. By setting the average to 6.2 we are 97.5% "confident" the pencil will be >6" (assuming we're happy with the 2.5% pencils that are too long!)

Quote: charliepatrick

Please forgive me if this starts out not being directly gambling, so here goes.

Suppose someone is making pencils, filling coke bottles, cans of food, whatever. It is almost impossible to make a pencil exactly 6" long. Typically there will be an average and, due to manufacturing, different length pencils will actually be produced. Most natural things approximate to the normal curve/distribution.

If we wanted to ensure that a certain percentage of pencils were 6" or longer (99% 95% or whatever), we would have to set the average length to be manufactured at more than 6". On the other hand, we might want to lower the variance (i.e. better quality production) and reduce the average error. The SD gives a measure of how far out we usually are. So for instance if it was .1", we might aim to produce pencils of average 6.2", so usually they would be 6.0" to 6.4". However if we were much better at manufacturing, we might be able to aim towards an average of 6.04".

Back to gambling. The usual question is each time I make a bet of (say) £1 what will happen. The average tells us that in the long term we'll get back (say) 98p per £ (or c per $ if you prefer). Obviously, normally, we'll either win multiples of £1 or nothing, and after (say) making 1000 bets we'll have won back something (or possibly nothing!) The SD gives an indication of how near £980 we're likely to be - like the pencils how far out we usually are. If the SD is large then we might make a lot of money, or have lost a lot. If the SD is low (such as even money roulette or punto banco bets) it is likely we'll be (say) between 900 and 1100 - so we would tend not to lose a lot nor win a lot - it's less "risky".

You may have heard risk associated with investments - essentially they are bets (albeit with positive expectations). A (good!) bank pays the same, called interest - little risk - no SD you know what you'll get back at end of year (X + Y%). A Dow Jones share has some risk - medium SD. A minor oil drilling company, or similar, might have major risk - the upsides are good - but the risk of losing everything is quite high! As they say you pays your money and takes your choice.

PS The reason for the 68% is that is the chance of being within 1SD.
In our exaggerated example 68% of pencils will be 6.1" to 6.3", 95% will be 6.0" to 6.4" (2 SDs), 99% will be 5.9" to 6.5" (3 SDs) etc. By setting the average to 6.2 we are 97.5% "confident" the pencil will be >6" (assuming we're happy with the 2.5% pencils that are too long!)

Good post in explaining what SD is, but I already know basically what it is. I just want to know how the numbers are used. Your last paragraph starts to talk about it I suppose. Like what does 1 SD mean. That it's a 68% chance it will be exactly the mean? So like I said in my previous post, why talk in SD, why not just talk in %? So instead of saying something is 1 SD, just say it has a 68% accuracy rate or something like that.

When the realm of data is known (Say, a Blackjack game of a given set of rules or a slot machine with the expected frequency of results known), the standard deviation helps identify how expected a given result is. I'll give an example that's less number-oriented, but I think it makes the point.

To wit, if I told you I lost 10 hands of BJ consecutively on XXX online gambling site and was whining that the site was rigged, you would all give me the finger and tell me to f--- off (some more politely than others ;)). You would all do this because, while unfortunate, a -10 streak in blackjack isn't terribly uncommon.

Now, if I told you (and properly documented) a 200-straight hand losing streak in that same BJ, people would start to raise eyebrows. Perhaps the game isn't actually being played as it's advertised. And the reason one might think that is because the results are just so extraordinarily unexpected in this game.

Similarly, if I told you I played 200 spins of a "Hundred-or-nothing" slot machine and didn't win once and made similar complaints, you would all tell me to shut up again. That result - even 200 losses in a row - is just downright expected on this game because it's more variant. It might take 1,000 straight losing (or even more) spins of this game before anyone reasonably suspected foul play.

On the flip side, the casinos leverage the same logic. If I hit 3 major jackpots in mere hours from each other, maybe I'm doing something that isn't on the up-and-up to "entice" these outcomes that would otherwise take weeks, months or years to happen. Or, maybe I'm just damned lucky. Just because something is significantly deviant from what's expected doesn't make it impossible. Merely, as Marvin the Paranoid Android would say, improbable.

So, back to your 2 video poker machines. Your machine with a SD of 20 will "win" more often, but the wins will be smaller. Over a stretch of time, you'll lose, lose, win, lose, win, win, lose, win, lose, lose, lose, lose, win, win... On the machine with an SD of 80, you won't win as often, but when you do, it'll be big wins: lose, lose, lose, lose, win, lose, lose, WIN, lose, lose, lose, lose, lose, lose, lose, lose, lose, win, lose, lose, lose, lose, lose, lose, lose, lose, lose, lose, OMFG WIN

Over a billion hands, these two machines might have the same house edge (the same as expected value or weighted mean of all possible results from the machine), but if you have a single Benny in your pocket and a limited number of hands to play, you have a choice. You can pick the machine that will probably let you play for a longer time, sacrificing the possibility of that big hit. Or you can play the machine that will probably drain you of your money pretty fast. But maybe, just maybe you'll be the one who will hit it big. Standard deviation differentiates these two machines.

Is that helpful?