Stories. Questions. Different perspectives.
Browse WAURE
← Advice & Questions
Education · Advice & Questions ·

How do we correct a class poster that turns a p-value into a probability our claim is true?

Education category photographEducation

Our statistics poster uses a worked example supplied for the class, with a p-value of 0.03. A teammate has written that this means there is a 97% chance our claim is true. It sounds clear, and none of us questioned it until I tried to explain where the 97% came from.

The American Statistical Association's statement says p-values do not give the probability that the hypothesis being studied is true. I can see that our headline needs changing, but I do not yet have a replacement I could confidently explain.

What should we check and say instead? I want to correct the interpretation without replacing one impressive sentence with another that we have simply copied from a guide.

Discussion

5 community replies

A p-value is calculated relative to a statistical model and its assumptions. A smaller value can signal a poorer fit between those assumptions and the observed data; it does not assign a probability to your claim. Find the model used in the exercise and check that explanation with the tutor.

Keep the observed difference in the example separate from the p-value. The ASA also distinguishes statistical significance from the size or practical importance of a result. Show whatever effect estimate the exercise supplies rather than inventing one from 0.03.

You could make the revision a group explanation task: each person should be able to say what the analysis examined and what the conclusion does not establish. A sentence that nobody can explain is not made useful by sounding technical.

Be careful with a replacement such as “there is only a 3% chance this happened by chance.” That still does not identify the statistical assumptions or accurately explain what probability is being calculated.

The missing assumptions are exactly what to bring back to the worked example. Check the tutor's explanation against the model used, then write a modest interpretation alongside the observed result. Neither 3% nor its complement can simply be relabelled as the probability your claim is false or true.

Join the discussion

More in Education