Calculate significance and power from a discrete critical region.
Notes and key methods
Under \(H_0:p=p_0\), use the exact binomial distribution with parameter \(\displaystyle p_{0}\). For an upper critical region \(\displaystyle X\ge c\), the actual significance is \(P_{p_0}(X\ge c)\), which need not equal the target level because X is discrete.
At a specified alternative \(\displaystyle p_{1}\), power is \(P_{p_1}(X\ge c)\), and the Type II probability is one minus this power. The data do not determine the null parameter; it is specified before testing.
Check the conditions
The alternative direction and critical region must match.
Power needs a particular alternative parameter.
Three worked examples
Example 1 · Exact binomial hypothesis tests
\(X\sim\operatorname{Bin}(16,p)\). Test \(H_0:p=0.25\) against a larger p, rejecting for \(X\ge 8\). For power use \(p=0.45\). The observed count is 9.
Use a 5% significance level for a written conclusion. A distribution calculator may be used for numerical p-values.
Find the actual significance of this critical region.
Find the power at the specified alternative.
Find the Type II error probability at that alternative.
Find the one-sided p-value when \(\displaystyle X=9\).
Show the reasoning
(a)
\(P_{0.25}(X\ge 8)=0.0271299562882632\)
Answer: \(0.0271299562882632\).
(b)
\(P_{0.45}(X\ge 8)=0.437100660250912\)
Answer: \(0.437100660250912\).
(c)
\(1-\operatorname{power}=0.562899339749088\)
Answer: \(0.562899339749088\).
(d)
\(P_{0.25}(X\ge 9)=0.00746972043998539\)
Answer: \(0.00746972043998539\).
The p-value is \(0.00746972043998539\). Since it is at most 0.05, reject \(\displaystyle H_{0}\): there is evidence that the success probability exceeds its null value. This does not give the probability that \(\displaystyle H_{0}\) is true.
Example 2 · Exact binomial hypothesis tests
\(X\sim\operatorname{Bin}(12,p)\). Test \(H_0:p=0.3\) against a larger p, rejecting for \(X\ge 7\). For power use \(p=0.5\). The observed count is 8.
Use a 5% significance level for a written conclusion. A distribution calculator may be used for numerical p-values.
Find the actual significance of this critical region.
Find the power at the specified alternative.
Find the Type II error probability at that alternative.
Find the one-sided p-value when \(\displaystyle X=8\).
Show the reasoning
(a)
\(P_{0.3}(X\ge 7)=0.038600843058\)
Answer: \(0.038600843058\).
(b)
\(P_{0.5}(X\ge 7)=0.38720703125\)
Answer: \(0.38720703125\).
(c)
\(1-\operatorname{power}=0.61279296875\)
Answer: \(0.61279296875\).
(d)
\(P_{0.3}(X\ge 8)=0.00948937113000001\)
Answer: \(0.00948937113000001\).
The p-value is \(0.00948937113000001\). Since it is at most 0.05, reject \(\displaystyle H_{0}\): there is evidence that the success probability exceeds its null value. This does not give the probability that \(\displaystyle H_{0}\) is true.
Example 3 · Exact binomial hypothesis tests
\(X\sim\operatorname{Bin}(13,p)\). Test \(H_0:p=0.2\) against a larger p, rejecting for \(X\ge 6\). For power use \(p=0.4\). The observed count is 7.
Use a 5% significance level for a written conclusion. A distribution calculator may be used for numerical p-values.
Find the actual significance of this critical region.
Find the power at the specified alternative.
Find the Type II error probability at that alternative.
Find the one-sided p-value when \(\displaystyle X=7\).
Show the reasoning
(a)
\(P_{0.2}(X\ge 6)=0.0300353232896\)
Answer: \(0.0300353232896\).
(b)
\(P_{0.4}(X\ge 6)=0.4256035766272\)
Answer: \(0.4256035766272\).
(c)
\(1-\operatorname{power}=0.5743964233728\)
Answer: \(0.5743964233728\).
(d)
\(P_{0.2}(X\ge 7)=0.00700356116480001\)
Answer: \(0.00700356116480001\).
The p-value is \(0.00700356116480001\). Since it is at most 0.05, reject \(\displaystyle H_{0}\): there is evidence that the success probability exceeds its null value. This does not give the probability that \(\displaystyle H_{0}\) is true.
Common mistakes
Do not calculate the null p-value using the observed proportion.
A requested 5% discrete test may have actual size below 5%.