Doohickey Question #2

I think "y" has the highest statistical confidence of having 95% positive reviews.

It would take longer to explain the methodology.than to do the mental calculation. I basically looked at the number of negative reviews you would expect from x,y, and z if their fraction of positive reviews were 95% , 90% and 85%. For each of the 9 cases I then compared the standard deviation of the expected number of negative reviews to the actual number of negative reviews and used that to judge something akin to statistical confidence in each of the 9 cases.

I found very little statistical confidence that X was 85%. 90% or 95% because of the small sample size. I judged it was the doohickey most likely to have only 85% positive reviews, because there seems to be more confidence in doohickeys Y and Z being either 90% or 95%. Its close between Y and Z, but I judge there is more statistical confidence that Z is 90% than there is that Y is 90%. Ergo, that leaves Y as the doohickey most likely to have 95% positive reviews.

I fully expect ThatDonGuy to have an elegant and lengthy statistical explanation as to why doohickey z is the correct one to choose and I look forward to reading it and learning from it. However, I did this analysis in my head, which is how most of us make decisions.

I don’t know how to make tables, so am not going to show all the steps of my work. Instead, I’ll just describe how I analyzed the problem, understanding that it may not make this beer worthy.

As an initial step, note that Y has the highest sample mean of 96% vs Z which is at 93% (186/200). And if Z were actually 95%, that would be a 1.2977 standard deviation difference from the mean. Not significant but not nothing compared to Y, which is only 0.324443 standard deviations from 95%. So my default would be Y.

But to analyze further, I calculated the p-score (using Excel two tailed distribution formula) for each of X, Y and Z being a sample drawn from a distribution of 85%, 90% and 95%. That gives the following (sorry for not being in table format):

p-scores (2 tail t-test):

85% / 90% / 95%
X 21.7% / 31.9% / 48.7%
Y 3.4% / 16.4% / 74.7%
Z 0.2% / 15.9% / 19.6%

From the table, the answer jumps out again with Y being so much more likely at 95% than 90% vs Z with a modest change in p-score.

But to combine properly note that there are 3! = six possibilities of which is 85%, 90% and 95%.

So combining those p-score chances and then grossing the raw probability up to 100% (because we are told that one of the orders must be correct), we get:

85/90/95
XYZ = 18.3%
XZY = 67.7%
YXZ = 5.6%
YZX = 7.0%
ZXY = 1.1%
ZYX = 0.4%

Again, pretty stark what the answer is.

Summing up the table, the chance that X Y or Z is the 95% positive distribution gives:

X = 7.3% (rounding)
Y = 68.8%
Z = 23.9%

So you should pick Y.

multiplying out the probabilities for the six permutations of 85,90,95, I get the most likely scenario for the samples shown is the xyz order of 85,95,90 respectively. So choose y