What is the setting?
From Holland and Rubin, On Lord's Paradox, 1983:
A large university is interested in investigating the effects on the students of the diet provided in the university dining halls and any sex differences in these effects. Various types of data are gathered. In particular, the weight of each student at the time of his [or her] arrival in September and his [or her] weight the following June are recorded.
The key elements of Lord's paradox
- The data: each student's weight in September and again in June, and each student's sex.
- The treatment: the dining-hall diet. Every student eats it, so no one is observed on any other diet.
- The question: did the diet affect weight, and did it affect men and women differently?
- Two standard analyses: one compares each group's change in weight from September to June; the other compares men's and women's June weights after adjusting for September weight.
- The paradox: applied to the same data, both analyses are computed correctly, yet they reach different conclusions.
- Your task: work out why, and what it would take to decide which conclusion, if either, to believe.
Make sure everyone in your group understands the setting. Raise your hand if you have questions. We will call the two analysts Statistician 1 and Statistician 2.
What does this page assume and leave out?
- Data: six invented students, three women and three men, from the Holland and Rubin setting.
- Models: Statistician 1 fits june - sept = a + b·female. Statistician 2 fits june = a + c·sept + b·female.
- Simplification: no noise, so every line fits exactly; no standard errors are read.
- Interpretation: the diet is the treatment; sex is an attribute, not a manipulable treatment.
- Style: plots use palette colors with different shapes in place of red and blue, so color is never the only cue.
Left out: sampling variability and formal methods for non-randomized data.
Setup
Question 1. What happened to the weights of three women between September and June?
Fill in the two gaps, then run. Group check: does everyone in your group get the same numbers?
Hint 1
Usemean() on a vector of weights.Hint 2
Subtract the vectors first,women_june - women_sept, then take the mean of the result.Question 2. What happened to the weights of three men?
Hint 1
Do the same as in Question 1, with the men's vectors.Hint 2
mean(men_june - men_sept) gives the average change.Do you agree with Statistician 1?
Statistician 1: No effect of diet on weight for either males or females, and no evidence of differential effect of the two sexes, because no group shows any systematic change.
Before scrolling further, decide as a group whether you agree.
Question 4. Where do the six students sit relative to the equal-weight line?
Women are circles, men are triangles. Points on the dotted line weighed the same in June as in September. Fill in the men's June weights, then count the students who gained and the students who lost.
Hint 1
The vector of men's June weights already exists in the setup.Hint 2
sum(c(women_june, men_june) > c(women_sept, men_sept)) counts the students above the dotted line; flip the sign to count those below it.Question 5. What does each group's line say about June weight at a September weight of 145?
Fit a line for the men like the one given for the women. The dot-dash vertical line marks 145 lb. Group check: agree who is predicted to weigh more in June.
Hint 1
Copy the patternlm(women_june ~ women_sept) with the men's variables.Hint 2
predict(reg_men, data.frame(men_sept = 145)); the column name must match the predictor in the model.Do you agree with Statistician 2?
Statistician 2: Compare June weight for males and females with the same weight in September (see the vertical line at 145 above): on average, for a given September weight, men weigh more in June than women. Thus, the new diet leads to more weight gain for men.
What is the causal set-up?
Answer in your group, in a sentence or two each.
Ready to see what happened?
Open this after Question 14.
Lord's paradox
Both statisticians computed correctly. Fitted to this data, Statistician 1's coefficient on female would be 0 and Statistician 2's would be -15. They disagree because each fills in the missing Y(0), the June weight without the dining-hall diet, with a different assumption.
Statistician 1 (change score) assumes each student would have stayed at their September weight. Statistician 2 (adjusting for September weight) assumes that without the diet, men and women with the same September weight would have had the same June weight, following one line of slope 0.5, so the 15 lb gap between the observed lines is credited to the diet. The data are identical and cannot tell which assumption is correct.
change score: june - sept = a + b*female adjusted: june = a + c*sept + b*female
This is Lord's paradox: both analyses target the same causal question, the difference between the diet's effect on women and on men, and the answer depends on an assumption about the counterfactual that the numbers cannot check.