Does a statistically significant result mean the improvement is important in practice?
EducationA made-up example in our statistics class reports a difference of 0.1 points on a 100-point test and labels it statistically significant. I expanded that into “a major educational improvement” in my summary.
The American Statistical Association's statement says statistical significance does not measure the size of an effect or the importance of a result. That makes me doubt the wording I chose.
What else should our summary examine? I do not want to dismiss a small difference automatically, but the significance label alone does not seem to answer whether changing the teaching approach would be worthwhile.