Stories. Questions. Different perspectives.
Browse WAURE
← Advice & Questions
Politics · Advice & Questions ·

The figures improved after a policy change. Can we say the policy caused the improvement?

Politics category photographPolitics

The practice chart shows more people using a town walking route after new signs were installed. These are illustrative results in a discussion exercise, not a report of a real programme.

My first caption credits the signs with the increase. Another member asked what else changed during the period and what would have happened without them. We only have the before-and-after comparison on the worksheet.

The government's impact-evaluation guidance distinguishes observing an outcome from attributing it to a policy. I don't want to describe the chart as useless, but I can see that my caption may claim more than it shows.

What can we say now, and what further evidence would help us assess the explanation rather than just the sequence?

Discussion

5 community replies

The chart shows a change over time. It doesn't by itself separate the booking change's effect from other possible influences. Describe the observed sequence before assigning a cause.

The guidance's counterfactual question is what would have happened without the intervention. A simple before-and-after comparison can give a weak answer to that, especially with only one observation on each side.

Check that the visits are measured consistently across the periods. If the definition or recording method changed, that would be another issue to resolve before interpreting the difference.

A stronger assessment doesn't always mean demanding a randomised trial for every small public-service change. The guidance calls for an evaluation design appropriate and proportionate to the question.

For a real assessment, ask what credible comparison or other evidence could strengthen the explanation and what assumptions it would require. For this exercise, keep the caption to the observed improvement and state that the cause hasn't been established by the information provided.

Join the discussion

More in Politics