Inference and Experiments
Introduced| English | Meaning |
|---|---|
| random assignment/ˈrændəm əˈsaɪnmənt/ | random assignment |
| random selection/ˈrændəm sɪˈlekʃn/ | random selection |
| statistically significant/stəˈtɪstɪkli sɪɡˈnɪfɪkənt/ | statistically significant |
| scope of inference/skəʊp ɒv ˈɪnfərəns/ | scope of inference |
Two kinds of randomness, two payoffs
- Statistics uses randomness in two different places, with two different rewards.
- Random assignment 随机分配 (to treatments) → lets you claim cause-and-effect.
- Random selection 随机选择 (of units from a population) → lets you generalize to that population.
- Knowing which one a study used tells you what its results can support.
Random assignment → causation
- Randomly assigning units to treatments balances all other variables across groups.
- So any difference in the response can be pinned on the treatment itself.
- This is why experiments — and only experiments — can prove cause.
- Without random assignment, a confounder could always be the real cause.
If random assignment alone rarely produces a difference this large, the result is statistically significant 统计显著
Random selection → generalization
- Randomly selecting units from the population makes the sample representative.
- So the conclusion generalizes to that whole population.
- Without random selection, results apply only to the units actually studied.
- Selection is about who's in the study; assignment is about what they get.
Significance and scope
- A difference is statistically significant if it's too large to be reasonably explained by chance.
- The scope of inference 推断范围 = what the two randomizations jointly permit.
- Random assignment + random selection → cause-and-effect that generalizes (the strongest scope).
- Only assignment → cause but not generalizable; only selection → generalizable association, not cause.
Two separate randomizations, two separate conclusions. Random assignment buys causation; random selection buys generalization. A study can have one, both, or neither — and its scope of inference is exactly what those two choices allow. Don't claim cause without random assignment, or generalization without random selection.
$60$ volunteers are randomly assigned to a new study method or the old one; scores rise significantly.
- Random assignment → the method caused the gain (a valid causal claim).
- But they were volunteers, not randomly selected → can't generalize to all students.
- Scope: cause-and-effect for these subjects, not a population-wide claim.
Random assignment to treatments licenses a cause-and-effect claim; random selection of units licenses generalization to the population. A difference is statistically significant when it's implausibly large for chance alone. The scope of inference is set by which randomizations a study used.
Write from random assignment and cause
- Students are randomly assigned to two study apps.
- The students were not randomly chosen from all schools.
What conclusion is supported?
Assignment and selection answer different questions.
What does the random assignment support?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Assignment supports cause. It does not create a random sample of schools.
Random assignment to treatments
Random assignment balances other variables across treatment groups.
Match each randomization to what it permits.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Assignment → causation; selection → generalization.
Volunteers are randomly assigned to two study methods and one wins significantly. You can conclude...
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Random assignment gives causation; volunteers block generalization.
A difference too large to be reasonably explained by chance is called statistically ___.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
That's the meaning of statistically significant.
Random selection of units from a population is what allows you to generalize the results to that population.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Random selection buys generalization.
A study with random assignment but NO random selection supports...
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Assignment → cause; without selection it doesn't generalize.