Stories. Questions. Different perspectives.
Browse WAURE
← Advice & Questions
Politics · Advice & Questions ·

Does random assignment in a policy trial mean the participants represent everyone?

Politics category photographPolitics

In a workshop example, volunteers sign up online for a trial of two versions of a council reminder letter. The volunteers are then randomly assigned to receive one version or the other. Someone describes the resulting groups as “a random sample of all residents.”

The Magenta Book’s evaluation-methods guidance describes random assignment as a way to construct comparison groups within a trial, while also discussing limits to generalising findings. Assignment to a group does not itself establish how people were recruited.

How should we explain the two stages without dismissing the trial? I want the workshop to distinguish a comparison among participating volunteers from a claim about how well those volunteers represent every resident. There are no actual trial results in this example.

Discussion

5 community replies

Draw two steps: who entered the study, then how participants were assigned. Putting “random” beside the second arrow does not explain the first.

Report what the design can address before describing the limit. It may support a useful comparison between the two letters among participants without establishing the same result for every resident.

Ask what information the example gives about recruitment. An online sign-up and a random selection from a population are different descriptions; don’t quietly replace one with the other.

Would highlighting recruitment make people conclude the random assignment was pointless?

It needn’t. Explain the comparison random assignment is intended to support, then identify the separate question of wider applicability. A finding can have a useful scope without answering every question about other people or settings.

Join the discussion

More in Politics