Module 6: Hypothesis Testing, p-values, Confidence Intervals & Statistical Power
Module 6: Hypothesis Testing, p-values, Confidence Intervals & Statistical Power
Formulating null (H0) and alternative (H1) hypotheses, interpreting p-values correctly according to ASA guidelines, Type I (alpha) and Type II (beta) errors, statistical power (1-beta), and 95% confidence intervals.
6. Foundations of Hypothesis Testing, p-values & Statistical Power
Statistical inference allows researchers to draw generalizable conclusions about an entire ecological population using a limited sample. Hypothesis testing evaluates whether observed differences in environmental data represent genuine biological phenomena or mere random sampling noise.
Null vs. Alternative Hypotheses:
Null Hypothesis ($H_0$): There is no real effect, difference, or association in the population ($\mu_{ ext{upstream}} = \mu_{ ext{downstream}}$). Any observed divergence in sample means is purely due to chance.
Alternative Hypothesis ($H_1$): There is a real effect or difference ($\mu_{ ext{upstream}} eq \mu_{ ext{downstream}}$).
What is a p-value (and what is it NOT)?
According to the American Statistical Association (ASA) Statement on $p$-values (Wasserstein & Lazar, 2016):
✓ Formal Definition: A $p$-value is the probability, assuming the null hypothesis ($H_0$) is true, of observing a test statistic at least as extreme as the one calculated from your sample.
- ✗ The $p$-value is NOT the probability that the null hypothesis is true.
- ✗ A $p < 0.05$ does NOT mean the effect is biologically or ecologically significant.
- ✗ A $p > 0.05$ does NOT prove that there is no difference ("absence of evidence is not evidence of absence").
Decision Errors: Type I ($\alpha$), Type II ($\beta$) & Statistical Power ($1 - \beta$)
| Statistical Decision | $H_0$ is Actually True (No Effect) | $H_0$ is Actually False (Real Effect Exists) |
|---|---|---|
| Reject $H_0$ (Claim Effect) | Type I Error (False Alarm, $\alpha = 0.05$) | Correct Discovery (Statistical Power = $1 - \beta$) |
| Fail to Reject $H_0$ | Correct Decision ($1 - \alpha = 0.95$) | Type II Error (Missed Discovery, $\beta$) |
Hands-On R Script: Prospective Power & Sample Size Calculation
# install.packages("pwr")
library(pwr)
# Research Question: How many river water samples (n) per site are required to
# detect a moderate difference in nitrate concentration (Cohen's d = 0.6) with
# standard power (80%) at alpha = 0.05?
power_calc <- pwr.t.test(
d = 0.6, # Expected effect size (mean diff / pooled SD)
power = 0.80, # 80% statistical power
sig.level = 0.05, # Type I error rate
type = "two.sample",
alternative = "two.sided"
)
print(power_calc)
# Result: n ≈ 45 samples per site required.
# Collecting only 10 samples would yield power < 25%, resulting in a massive Type II error risk!Common Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.