Does random assignment in a policy trial mean the participants represent everyone?
PoliticsIn a workshop example, volunteers sign up online for a trial of two versions of a council reminder letter. The volunteers are then randomly assigned to receive one version or the other. Someone describes the resulting groups as “a random sample of all residents.”
The Magenta Book’s evaluation-methods guidance describes random assignment as a way to construct comparison groups within a trial, while also discussing limits to generalising findings. Assignment to a group does not itself establish how people were recruited.
How should we explain the two stages without dismissing the trial? I want the workshop to distinguish a comparison among participating volunteers from a claim about how well those volunteers represent every resident. There are no actual trial results in this example.