Differential Expression Modeling with DESeq2 and edgeR
Differential Expression Modeling with DESeq2 and edgeR
Formulating the Negative Binomial model (Y_ij ~ NB(mu_ij, alpha_i)), empirical Bayes dispersion shrinkage, Wald tests, and log2 fold-change shrinkage using apeglm and ashr.
Key Learning Deliverables & Competencies
- Theoretical breakdown of the Negative Binomial distribution vs Poisson for RNA-Seq count overdispersion
- Empirical Bayes dispersion estimation and variance stabilization (vst & rlog)
- Pairwise contrast extraction: Drought-Tolerant vs Susceptible under progressive drought stages
- Effect-size moderation: apeglm and ashr shrinkage of noisy low-count fold changes
1. Why the Negative Binomial Distribution is Mandatory
Technical replicate counts follow a Poisson distribution, where the variance equals the mean ($\sigma^2 = \mu$). However, biological replicates in agricultural and plant studies incorporate genetic, micro-environmental, and developmental stochasticity. Biological variance exceeds the mean: this phenomenon is called overdispersion.
The Negative Binomial distribution models count data with an overdispersion parameter $\alpha_i$: \[ K_{ij} \sim \text{NB}(\mu_{ij}, \alpha_i), \quad \text{Var}(K_{ij}) = \mu_{ij} + \alpha_i \mu_{ij}^2 \] As $\alpha_i \to 0$, the Negative Binomial converges to the Poisson distribution. DESeq2 uses an Empirical Bayes shrinkage procedure that shrinks individual gene dispersion estimates toward a smooth mean-dispersion trend curve.
2. Executing Differential Expression with DESeq2
library(DESeq2)
library(apeglm)
# Run complete DESeq2 pipeline
# (estimates size factors, dispersions, and fits Generalized Linear Models)
dds <- DESeq(dds)
# Inspect dispersion plot to verify biological shrinkage
plotDispEsts(dds, main = "Solanum tuberosum Dispersion Estimates")
# Extract contrast: Tolerant Genotype Severe Drought (D7) vs Control (D0)
res_tol_d7 <- results(
dds,
contrast = c("Drought_Stage", "D7_Severe", "D0_Control"),
alpha = 0.05
)
# Apply apeglm LFC shrinkage to prevent false high fold-changes in low-count genes
res_tol_d7_shrunk <- lfcShrink(
dds,
coef = "Drought_Stage_D7_Severe_vs_D0_Control",
type = "apeglm"
)
# Summarize results
summary(res_tol_d7_shrunk)Common Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.