Examqo › Examqo library › Public health › Biostatistics and screening · IMLE
Hypothesis testing, P-values and confidence intervals
A P-value is the probability of a result at least as extreme as the one observed if the null hypothesis is true, and a confidence interval shows the range of plausible true values – both answer "is this chance?", not "does this matter?".
Why it matters in the exam
Almost every exam has definition items – what a P-value is, what a 95% confidence interval means, what type I and type II errors are – and sample-size items: what happens to the required sample size if α is reduced, power is raised or a smaller difference is sought. Other items show a relative risk with a confidence interval that includes 1 and ask whether the finding is significant, or ask you to distinguish statistical significance from clinical importance. Also tested are multiple comparisons, which raise the chance of a false-positive finding, and one-sided versus two-sided tests. The definition used in many items is that if the study were repeated many times and a 95% confidence interval built each time, about 95% of the intervals would contain the true value. Remember too that a non-significant result in a small study does not prove there is no difference; it may only reflect low power.
Key points
- A P-value is calculated assuming the null hypothesis is true.
- A confidence interval that includes the null value indicates no significance.
- The usual α is 0.05 and the usual power is 80%.
- A smaller α, higher power or a smaller difference requires a larger sample.
- Multiple comparisons raise the risk of a false positive.
Summary table
| Concept | Definition | Usual value | What affects it |
|---|---|---|---|
| Type I error (α) | Rejecting a true null hypothesis – a false positive | 0.05 | A smaller α increases the required sample |
| Type II error (β) | Failing to reject a false null hypothesis – a false negative | 0.2 | A smaller β increases the required sample |
| Power (1 minus β) | The chance of detecting a difference that truly exists | 80% or 90% | Rises with a large sample, a large difference and small spread |
| P-value | Probability of a result at least as extreme as observed under the null hypothesis | Below 0.05 counts as significant | It is not the probability that the null hypothesis is true |
| 95% confidence interval | A range built so that in 95% of repetitions it contains the true value | If it includes 1 (for a ratio) or 0 (for a difference) – not significant | Narrower in a large sample |
A free exam question
אורך בדיקת קורונה מתפלג נורמלי עם תוחלת 3 דקות. במשרד הבריאות חושדים שבמתחם בדיקות מסוים תוחלת משך הבדיקה שונה מהרגיל. רווח- סמך של 95% שנבנה על סמך מדגם מקרי של בדיקות ) :ממתחם זה 3.04,3.55 .( מה ניתן להסיק מרווח זה לגבי השערת משרד הבריאות, ללא חישוב נוסף? (the question as examined, in Hebrew)
- א. ברמת מובהקות 5% - טענתם נכונה; ברמת מובהקות 2.5% - לא ניתן לדעת
- ב. ברמת מובהקות 5% - טענתם לא נכונה; ברמת מובהקות 2.5% - טענתם לא נכונה
- ג. ברמת מובהקות 5% - טענתם לא נכונה; ברמת מובהקות 2.5% - לא ניתן לדעת
- ד. ברמת מובהקות 5% - טענתם נכונה; ברמת מובהקות 2.5% - טענתם נכונה
In the full version for subscribers
- A simple algorithm
- Memory aid
- Exam traps
- 5 review cards
- Enough for IMLE
The full version is open to subscribers of: Public health, Medical licensing exam (IMLE).
Subscriber? Read the full topic Plans and prices 25 free questions
Sources
- Kirkwood & Sterne, Essential Medical Statistics, 2nd ed., 2003 — Confidence interval and P-value; Calculation of required sample size
- Wasserstein & Lazar, The ASA statement on p-values, 2016 — The American Statistician – principles of interpretation
Sources are given for further reading; the text is written afresh and not copied from them.
Reviewed by the Examqo AI Professor · 2026-11-06. The text may contain errors; when in doubt, check the source.