Inference for Categorical Data: Proportions

A poll says $52\%$ of voters back a candidate, "with a margin of error of $3$ points". Is the candidate really ahead? A company claims $90\%$ of customers are satisfied, and your sample finds $84\%$. Is the claim false, or was your sample unlucky? This unit gives the two tools that answer such questions: confidence intervals and significance tests.

Because a sample proportion 样本比例 $\hat{p}$ is approximately normally distributed 正态分布的 (when the conditions 条件 hold), we can measure how far a sample result is from a claimed value in standard errors 标准误, and turn that into a probability. This is what makes inference 推断 – drawing conclusions about a population 总体 from a sample – possible.

A confidence interval 置信区间 estimates the parameter 参数 as a range: statistic $\pm$ margin of error 误差幅度.

$z^{*}$ is the critical value 临界值 for the confidence level 置信水平 (e.g. $1.96$ for 95%). Conditions: random sample 随机样本, Large Counts condition 大计数条件 ($n\hat p\ge 10$ and $n(1-\hat p)\ge 10$), and the 10% condition. Interpret it: “We are 95% confident the true proportion of... is between... and...”. Interpret the level: “In 95% of samples, this method produces an interval that captures the true proportion.”

Why the conditions matter. The interval assumes the sampling distribution of $\hat p$ is approximately normal, which is what makes $z^*$ the right multiplier. With fewer than $10$ successes or failures, that distribution is skewed and the interval does not capture $p$ as often as claimed. The random condition lets you generalize to the population; the 10% condition keeps the observations close to independent, so the standard error formula is correct.

Worked example. In a random sample of $200$ people, $120$ support a policy, so $\hat{p}=0.60$. A $95\%$ interval uses $z^*=1.96$:

We are $95\%$ confident the true proportion of supporters is between $53.2\%$ and $66.8\%$.

Common mistakes. Saying “there is a $95\%$ chance that $p$ is between $0.532$ and $0.668$”: $p$ is fixed, and this interval either contains it or does not; the $95\%$ describes the method. Describing individuals or the sample instead of the population proportion. Skipping the Large Counts check: here $n\hat p=120$ and $n(1-\hat p)=80$, both at least $10$.

Choosing the sample size. To keep the margin of error no larger than a target $m$, set $z^{*}\sqrt{\dfrac{\hat p(1-\hat p)}{n}}\le m$ and solve for $n$. When you have no estimate of $\hat p$, use $\hat p=0.5$: it makes $\hat p(1-\hat p)$ as large as possible, giving the safe (largest) required sample size. Always round the result up to the next whole person.

Worked example. How many people must you survey for a $95\%$ interval with margin of error at most $0.03$? Using $\hat p=0.5$ and $z^*=1.96$:

so you survey $1068$ people (always round up, since $1067$ would leave the margin a shade too big).

Width. The width of the interval is twice the margin of error. With everything else fixed, a larger sample gives a narrower interval (four times the sample halves the width), and a higher confidence level gives a wider one ($99\%$ uses $z^*=2.576$).

To judge a claimed value: if it lies inside the interval, the data are consistent with it; if it lies outside, the data give evidence against it. Base the conclusion on whether the plausible 可信 values include the claim, in context 结合情境.

Worked example (a discount program). A store's program should give a discount on $20\%$ of bills. A random sample gives the $95\%$ interval $0.15\pm0.06$, that is $(0.09,\ 0.21)$.

• Is there convincing evidence the program is not working as intended? No: $0.20$ is inside the interval, so $20\%$ is a plausible value.
• Is there convincing evidence that it does give the discount with probability $0.20$? Also no: many other values, such as $0.10$, are plausible too. An interval can never prove a single value.
• A second sample four times as large also gives $\hat p=0.15$. Its margin of error is half as big, $0.03$, so the interval is $(0.12,\ 0.18)$. Now $0.20$ is outside: there is convincing evidence that the program gives fewer discounts than intended.

Worked example (an interval in other units). A $95\%$ interval for the proportion of water-cup customers who take soft drinks is $(0.24,\ 0.38)$. Each such customer costs $\$0.25$, and $3000$ customers ask for water cups in June. Multiply both ends: $3000\times0.25\times(0.24,\ 0.38)=(\$180,\ \$285)$. We are $95\%$ confident the June cost is between $\$180$ and $\$285$.

A significance test 显著性检验 weighs evidence against a claim. State a null hypothesis 原假设 $H_0$ and an alternative hypothesis 备择假设 $H_a$ about the parameter $p$:

Check the same conditions (random, Large Counts using $p_0$, 10%). The test statistic 检验统计量 counts standard errors from $p_0$:

Worked example. A company claims $90\%$ satisfaction ($p_0=0.90$); a sample of $100$ finds $84$ satisfied ($\hat{p}=0.84$). Test $H_0:p=0.90$ vs $H_a:p\neq0.90$ at $\alpha=0.05$:

giving a two-tailed 双尾 $p$-value of about $2(0.023)=0.046$. Since $0.046<0.05$, we reject 拒绝 $H_0$ – there is evidence the true satisfaction rate differs from (is below) $90\%$.

The $p$-value P值 is the probability of getting a sample result as extreme or more extreme than the observed 观测到的 one, assuming $H_0$ is true. A small $p$-value means the data would be surprising if $H_0$ held – evidence against $H_0$. It is not the probability that $H_0$ is true.

A full interpretation has three parts: the assumption, the probability, and the observed result, in context. For the satisfaction test: "Assuming the true satisfaction rate is $90\%$, there is a $0.046$ probability of getting a sample proportion at least as far from $0.90$ as $0.84$ (in either direction) by chance in a random sample of $100$ customers." The distribution of the test statistic when $H_0$ is true is the null distribution; it may come from the normal model or from a simulation.

Common mistakes. Writing hypotheses about the statistic ($H_0:\hat p=0.90$): hypotheses are always about the parameter $p$. Using $\hat p$ in the test's standard error: the test assumes $H_0$, so it uses $p_0$ (the interval uses $\hat p$). Reading $p=0.046$ as “a $4.6\%$ chance that $H_0$ is true”.

Compare the $p$-value to the significance level 显著性水平 $\alpha$ (often $0.05$):

• $p\le\alpha$: reject $H_0$ – there is convincing evidence 有力证据 for $H_a$.
• $p>\alpha$: fail to reject 不拒绝 $H_0$ – not enough evidence for $H_a$ (never “accept $H_0$”).

Always write the conclusion in context, linking back to the claim.

Common mistakes. Writing “accept $H_0$” or “this proves $p=0.90$” after a large $p$-value: failing to reject only means the evidence is not strong enough. Concluding without the comparison of the $p$-value with $\alpha$, or without context.

• A Type I error 第一类错误: rejecting a true $H_0$ (a false alarm 误报). Its probability is $\alpha$.
• A Type II error 第二类错误: failing to reject a false $H_0$ (a missed detection 漏检). Its probability is $\beta$.
• The power 检验效能 $=1-\beta$ is the chance of correctly detecting a real effect 效应. Power rises with a larger sample, a larger effect, or a larger $\alpha$.

Describe each error and its consequence 后果 in the problem's context. Which error is worse depends on the situation, and that decides $\alpha$. If a Type I error is very costly (approving a drug that does not work), choose a small $\alpha$, such as $0.01$. If a Type II error is worse (missing a dangerous fault), a larger $\alpha$ such as $0.10$ raises the power.

Worked example. In the satisfaction test above ($H_0:p=0.90$):

• Type I error: concluding that the satisfaction rate is not $90\%$ when it really is $90\%$. Consequence: the company is wrongly accused of overstating its rate.
• Type II error: failing to find evidence against $90\%$ when the true rate is different (say $85\%$). Consequence: customers keep trusting a rate that is too high.

A good answer names the error and says what happens because of it, in the words of the problem.

Power in a picture. Test $H_0:p=0.50$ against $H_a:p>0.50$ with $n=100$ and $\alpha=0.05$. The test rejects $H_0$ when $\hat p\ge0.582$. If the true proportion is really $0.60$, the chance of rejecting is the area of the orange curve beyond $0.582$: the power is $0.64$, so $\beta=0.36$.

With a larger sample, both curves get narrower, so they overlap less: the power increases and $\beta$ falls. Power also increases with a larger $\alpha$ (the cut-off moves left) and with a true value farther from $p_0$.

To compare two proportions, estimate $p_1-p_2$:

Conditions must hold in both samples, and the samples must be independent.

Worked example. For the two teaching methods below ($\hat p_1=\tfrac{45}{75}=0.60$, $\hat p_2=\tfrac{32}{80}=0.40$), a $95\%$ interval for $p_1-p_2$ is

which is $(0.046,\ 0.354)$. We are $95\%$ confident that method 1's true pass rate is between $4.6$ and $35.4$ percentage points 百分点 higher than method 2's. The interval uses the separate $\hat p$ values; the pooled $\hat p_c$ belongs only to the test.

If the interval for $p_1-p_2$ contains $0$, the data are consistent with no difference; if it lies entirely above or below $0$, there is evidence of a difference (in that direction). State the direction and context.

Hypotheses compare the two proportions: $H_0: p_1=p_2$ versus $H_a: p_1\neq p_2$ (or $<,>$). Because $H_0$ says the proportions are equal, use a combined (pooled) 合并 sample proportion $\hat p_c=\dfrac{\text{total successes}}{\text{total sample size}}$ to estimate the common $p$.

The pooled two-proportion $z$ statistic:

Find the $p$-value from the normal model, compare to $\alpha$, and conclude in context – the same four-step logic as the one-proportion test.

Worked example: a two-proportion test. Two teaching methods are compared with random samples of students. Method 1: 45 of 75 pass. Method 2: 32 of 80 pass. Is there convincing evidence, at $\alpha=0.05$, of a difference in pass rates?

• State. $H_0:p_1=p_2$ and $H_a:p_1\neq p_2$, where $p_1,p_2$ are the true pass rates under each method.
• Plan. Two-sample $z$-test for a difference of proportions. Independent random samples; each is less than 10% of its population; the counts $45,\ 30,\ 32,\ 48$ are all at least 10.
• Do. $\hat p_1=0.60$, $\hat p_2=0.40$, pooled $\hat p_c=\dfrac{77}{155}=0.497$.
  $$z=\frac{0.60-0.40}{\sqrt{0.497(0.503)\left(\frac{1}{75}+\frac{1}{80}\right)}}=\frac{0.20}{0.0804}=2.49,\qquad p\approx0.013.$$
• Conclude. Because $0.013<0.05$, reject $H_0$. There is convincing evidence that the true pass rates under the two methods differ; method 1's is higher.

Answer each before reading the answers below.

1. In a random sample of 160 students, 64 walk to school. Construct a $95\%$ confidence interval for the true proportion who walk.

2. How large a sample gives a $95\%$ margin of error of at most $0.04$, with no prior estimate of $p$?

3. A coin is tossed 400 times and lands heads 220 times. Test $H_0:p=0.5$ against $H_a:p\neq0.5$.

4. Which is the correct meaning of “$95\%$ confident” for the interval in question 1? (a) $95\%$ of students walk between $32\%$ and $48\%$ of the time. (b) The method used captures the true proportion in about $95\%$ of samples.

5. A student says: “My interval $(0.532,\ 0.668)$ has a $95\%$ chance of containing $p$.” Correct the student.

6. A $95\%$ interval for the proportion of a school's students who read daily is $(0.31,\ 0.43)$. The principal claims that $40\%$ read daily. Does the interval give convincing evidence against the claim? Does it prove the claim?

7. In a test of $H_0:p=0.30$, describe a Type II error in context if $p$ is the proportion of a company's employees who work from home.

8. How would increasing the sample size from $100$ to $400$ change the power of the test in the power figure? Explain.

1. $\hat p=0.40$, $SE=\sqrt{0.40(0.60)/160}=0.0387$, so $0.40\pm1.96(0.0387)=0.40\pm0.076$, giving $(0.324,\ 0.476)$.

2. $n=\dfrac{1.96^2(0.5)(0.5)}{0.04^2}=600.25$, so survey 601 people.

3. $\hat p=0.55$, $z=\dfrac{0.55-0.50}{\sqrt{0.25/400}}=\dfrac{0.05}{0.025}=2.0$, two-sided $p\approx0.046$. At $\alpha=0.05$, reject $H_0$: convincing evidence that the coin is not fair.

4. (b). The confidence level describes the method's long-run success rate.

5. Once computed, the interval either contains $p$ or it does not. The $95\%$ describes the method: about $95\%$ of intervals made this way capture $p$. Say “we are $95\%$ confident”, not “a $95\%$ probability”.

6. No: $0.40$ is inside the interval, so it is plausible. But the interval does not prove $p=0.40$, because values such as $0.35$ are plausible too.

7. A Type II error would be failing to find convincing evidence that the proportion who work from home differs from $0.30$ when it really does differ.

8. The power would increase. With $n=400$ the standard deviation of $\hat p$ halves, so the null and true sampling distributions overlap less, and a true value of $0.60$ is much more likely to give a $\hat p$ beyond the cut-off.

• State the conditions (random, 10%, large counts $np,\,nq\ge10$) before any proportion inference.
• A confidence interval = estimate $\pm$ margin of error; “95% confident” refers to the method's long-run capture rate.
• For a test, write $H_0$ and $H_a$, compute the test statistic, find the p-value, and compare to $\alpha$.
• A small p-value is evidence against $H_0$; failing to reject does not prove $H_0$.
• Larger samples shrink the margin of error; a higher confidence level widens it.
• A claimed value inside the interval is plausible, not proven; a value outside it is evidence against the claim.
• Interpret a $p$-value with all three parts: "assuming $H_0$ is true", the probability, and "a result at least as extreme as the one observed", in context.