Educerie · SAT · Math
Problem-Solving and Data Analysis · PSDA.7 Evaluating statistical claims: observational studies and experiments
What you must be able to do
| You must be able to | What it looks like on the test |
|---|---|
| Tell an experiment from an observational study | Were subjects assigned, or just watched? |
| Say what random assignment licenses | A conclusion about cause |
| Say what random selection licenses | A conclusion about a wider population |
| Reject conclusions the design cannot support | Three of the four options |
| Name the missing ingredient | …because subjects were not randomly assigned |
1The idea in one paragraph
Two different randomisations do two different jobs, and almost every question in this unit turns on keeping them apart. Random selection from a population lets you generalise to that population. Random assignment to treatment groups lets you claim cause. A study with both supports a causal claim about that population; a study with neither supports a description of the people studied and nothing more.
Observed, not assigned → no causal claim. Volunteered, not selected → no generalisation. Learn those two sentences and this unit is finished.
2The two-by-two that answers everything
| Random assignment | No random assignment | |
|---|---|---|
| Random selection | Cause, and generalisable | Association, generalisable |
| No random selection | Cause, within this group | Association, within this group |
That table is the entire skill. Read the study description twice — once asking how subjects were chosen, once asking how groups were formed — and the cell tells you what the correct answer may claim.
3Observational studies
The researcher watches and records. Nothing is imposed. These can show a strong association and can never, on their own, show cause, because the groups may differ in ways nobody measured.
Students who eat breakfast score higher — the breakfast eaters may also sleep more, or live in homes where breakfast is possible. Those are confounding variables, and the phrase appears in correct answers.
4Experiments
The researcher assigns subjects to groups. If that assignment is random, the groups are alike on average in every respect — measured and unmeasured — so a difference in outcome can be attributed to the treatment.
That is why random assignment, and only random assignment, licenses the word cause.
5Reading the wording of the options
Options are written to be almost right. The four levels, from weakest to strongest:
- In this sample, X and Y were associated. — almost always defensible.
- Among this population, X and Y are associated. — needs random selection.
- X causes Y in this group. — needs random assignment.
- X causes Y generally. — needs both.
Match the option's level against the design. The most common wrong answer is level 3 offered for an observational study.
6The limitation questions
Which is the most serious limitation of the study? Look for the missing randomisation first: no random assignment when a causal claim is being made, no random selection when the conclusion names a wider group. Sample size is rarely the answer, and it is never the answer when a randomisation is missing.
Where points are lost
- Reading an association as a cause because the study is large or the effect is strong.
- Mixing up the two randomisations.
- Generalising from volunteers, who selected themselves.
- Blaming sample size when the real problem is selection.
- Accepting a conclusion about a different population from the one studied.
- Ignoring a confounding variable that the description has quietly put in front of you.
Work it right
- Ask how the subjects were obtained: random selection, or not?
- Ask how the groups were formed: random assignment, or not?
- Place the study in the two-by-two.
- Grade each option: sample, population, cause, or cause generally.
- Choose the strongest option the design supports, and no stronger.
Try it
Q1. Researchers recorded the sleep and exam results of 500 students who volunteered, and found that students who slept longer scored higher. Which conclusion is best supported?
A) Sleeping longer causes higher exam scores. B) Among these 500 students, longer sleep was associated with higher scores. C) Sleeping longer causes higher scores for all students nationally. D) Exam scores cause students to sleep longer.
Q2. 300 patients were randomly selected from a hospital's records and randomly assigned to a new therapy or the standard one. The new therapy group recovered faster. Which conclusion is best supported?
A) The new therapy causes faster recovery among patients at this hospital. B) The new therapy is associated with faster recovery, but no cause can be claimed. C) The new therapy causes faster recovery for everyone. D) Nothing can be concluded because the sample is too small.
Q3. A gym emails its members a survey about exercise; 200 reply and 84% report exercising daily. What is the most serious limitation?
A) The sample size is small. B) Those who chose to reply may differ systematically from those who did not. C) The survey did not report a margin of error. D) Daily exercise was not defined precisely.
Q4. Which design is needed to support a claim that a fertiliser causes higher yield?
A) Observing fields that already use it B) Randomly assigning plots to fertiliser and no fertiliser C) Surveying farmers about their yields D) Comparing this year's yields with last year's
Q5. A study randomly assigns volunteers to two exercise programmes and finds a difference. What limits the conclusion?
A) No cause can be claimed. B) The result cannot be generalised beyond people like the volunteers. C) The margin of error is unknown. D) The groups were not comparable.
In one breath
Two randomisations do two jobs: random assignment to groups is what lets you say cause, and random selection from a population is what lets you say these people are like that population. An observational study shows association and never cause, however strong the pattern, because the groups may differ in ways nobody measured — and when a study asks about its most serious limitation, look for the missing randomisation before you blame the sample size.
Answers
Q1. B. volunteers, no assignment — association within the sample A and C claim cause from an observational study. D reverses the direction with no more support than A had.
Q2. A. random assignment gives cause; random selection from this hospital gives this population B understates a genuine experiment. C stretches beyond the hospital's patients. D blames a sample size that is ample.
Q3. B. non-response bias The people who answer a voluntary survey about exercise are likely to be the ones who exercise. A is the reflex answer and is not the problem here; C and D are real but minor beside self-selection.
Q4. B. only random assignment licenses cause The others all describe observation.
Q5. B. assignment was random, selection was not Cause can be claimed within the study; generalisation cannot, because volunteers chose themselves. A and D ignore the random assignment.
Educerie · written from the published College Board* Assessment Framework for the Digital SAT Suite *(Math, Problem-Solving and Data Analysis, skill/knowledge testing point "Evaluating statistical claims: observational studies and experiments"). All questions and explanations are original Educerie text. Last reviewed 12 September 2026.
Check your understanding
The main ideas of this note. Tick each one you could do now, in an exam, without looking back up. Anything you cannot tick yet is the part to read again.