Standard deviation, concept to be retired?

Standard Deviation
link

The notion of standard deviation has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of mean deviation.

Please quote everything:
"Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems."

"And the probability-related problems with social and biological science do not stop there: it has bigger problems with researchers using statistical notions out of a can without understanding them"

Definitely, the author is advocating a second-quality type of science for the vulgum pecus ("growing class of people") while first-class concepts "should be left to mathematicians, physicists and mathematical statisticians ". I profoundly disagree. Everybody is entitled to quality.

What is needed is better statistical education (I know, I preach for my chapel...)

They should really stop using it before someone loses an eye!

Quote: FleaStiff

The notion of...has confused hordes of scientists.

I never got Quantum Mathematics/Mechanics, so there's lots of confusing things around.

I think in statistics, and especially in gambling discussions, we need a definitive measure to describe not only the expected payback (i.e. average HE) but also how freaky the possible results are. For instance playing Even money bets has a low value, betting Jackpot type bets has a higher one.

Standard Deviation also answers the types of question where you want to know how likely an event will be off the norm and set a confidence of the outcome or set a test against a hypothesis.

Suppose you think a die might be biassed, then you can define M and N for a test where you will throw the dice M times and the test gets you N sixes; what value of N (for a given M) should you set [before the test] to be 99.9% sure the die is biassed. As M becomes large, this is based on the number of Standard Deviations - it says the chances of this happening with a true die are so low that we can assume the die must be biassed.

I'm getting a broken link. :(

Edit: found it by going to the main page, no worries.

This is a debate which has raged in math ed for a long time (like pi versus tau). Personally, I don't care. I'm a math Phd so suffice to say I'm familiar with the ups and downs of both. The way it is taught now makes sense, but a change to "MAD," or as I call it the 1 norm, may be better for learning. I would say the advantages are small, but likely there, and making math more intuitive for the masses is worthwhile. Math folks like me can worry about using the proper moments and measures skewness and kurtosis.

It is a broken link to me too.

Quote: Keyser

They should really stop using it before someone loses an eye!

Exactly. SD should come with a warning label. Use
at your own risk.

Quote: Kickass

It is a broken link to me too.

go here and search "Taleb" or "Standard deviation"
http://www.edge.org/annual-questions

Quote: endermike

This is a debate which has raged in math ed for a long time (like pi versus tau). Personally, I don't care. I'm a math Phd so suffice to say I'm familiar with the ups and downs of both. The way it is taught now makes sense, but a change to "MAD," or as I call it the 1 norm, may be better for learning. I would say the advantages are small, but likely there, and making math more intuitive for the masses is worthwhile. Math folks like me can worry about using the proper moments and measures skewness and kurtosis.

Frankly, when presenting the concept of dispersion parametre, I begin by introducing the students to the "more intuitive" concept of mean absolute deviation. But as quickly I point to them how unwieldy the concept is (with absolute values and the like), then I go to the door and make a gig of putting the same pressure close to the hinge or the edge. "See? The farthest from the pivot, the more effect! Similarly, large deviations receive more weight in the standard deviation concept." They get that SD is more close to physical concepts like the moment of inertia. I finish the presentation with comments about the natural appearance of SD and variance in Gaussian contexts (which of course I introduce with a different name...).

Taleb is the specialist of "Black Swans", i.e. exceptional results. In his domain, there is certainly a reason for using SD or MAD according to the needs. Not enough to call for the 'retirement' of SD :-)