Module 5: Probability Distributions, Normality Diagnostics & Mathematical Transformations
Tested Stacks:
GATK 4.6.1
Samtools 1.21
Nextflow 24.10
Seurat v5.1
Python 3.12
Prerequisites: Basic Linux CLI & Sequencing Formats
Module 05 • Normality & Transformations
Track 19 • Applied Environmental Statistics
Hands-On R & RStudio Included
Module 5: Probability Distributions, Normality Diagnostics & Mathematical Transformations
Gaussian normal distribution, testing assumptions with Quantile-Quantile (Q-Q) plots and Shapiro-Wilk test, homoscedasticity diagnostics (Levene test), and mathematical transformations (log10, sqrt) for skewed environmental metrics.
5. Probability Distributions, Normality & Mathematical Transformations
Most classical parametric tests (Student's $t$-test, ANOVA, linear regression) require the data or model residuals to follow a Gaussian Normal Distribution and display Homoscedasticity (equal variances across groups). Environmental pollutant concentrations (mercury, nitrate, bacteria CFU) are almost never normally distributed—they are typically positively skewed with long right-hand tails.
How to Assess Normality in R:
- Visual Assessment (Q-Q Plot): The Quantile-Quantile plot graphs empirical quantiles against theoretical normal quantiles. Points conforming tightly to the diagonal line indicate normal distribution. S-shaped curves indicate heavy tails; U-shaped curves indicate skewness.
- Statistical Testing (Shapiro-Wilk Test): Tests $H_0$: "Data is normally distributed" vs $H_1$: "Data is not normal". If $p < 0.05$, reject normality.
- Testing Homogeneity of Variance (Levene's Test): Tests whether variances are equal across treatment groups (
car::leveneTest()). If $p < 0.05$, equal variance assumption is violated.
Mathematical Transformations for Skewed Environmental Data
| Transformation | Mathematical Formula | When to Apply | R Syntax |
|---|---|---|---|
| Logarithmic | $y = \log_{10}(x)$ or $\log(x + 1)$ | Severe positive (right) skew; lognormal contaminant concentrations | log10(x) or log1p(x) |
| Square Root | $y = \sqrt{x}$ | Moderate positive skew; count data (macroinvertebrates, seeds) | sqrt(x) |
| Arcsine Square Root | $y = \arcsin(\sqrt{p})$ | Proportions or percentage data bounded strictly between 0 and 1 | asin(sqrt(p)) |
Hands-On R Script: Testing & Correcting Non-Normality
# 1. Simulate Skewed E. coli Contamination (CFU / 100 mL)
set.seed(99)
e_coli_raw <- rlnorm(40, meanlog = 3.5, sdlog = 1.1)
# 2. Shapiro-Wilk Normality Test on Raw Data
sw_raw <- shapiro.test(e_coli_raw)
print(sw_raw)
# Notice: p < 0.001 -> Strongly Rejects Normality!
# 3. Log10 Transformation
e_coli_log <- log10(e_coli_raw)
# 4. Shapiro-Wilk Test on Transformed Data
sw_log <- shapiro.test(e_coli_log)
print(sw_log)
# Notice: p = 0.68 -> Data is now completely consistent with normal distribution!
# 5. Visual Q-Q Plots
par(mfrow = c(1, 2))
qqnorm(e_coli_raw, main = "Raw E. coli (Severe Skew)"); qqline(e_coli_raw, col = "red")
qqnorm(e_coli_log, main = "Log10 E. coli (Normalized)"); qqline(e_coli_log, col = "blue")
par(mfrow = c(1, 1))
⚠️
Common Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.
Was this guide helpful to your research?
Help us maintain 100% accurate, peer-reviewed bioinformatics standards.