Curriculum Modules
Click on any chapter below to explore complete code, architecture diagrams, and executor profiles.
1: Workflow Orchestration, Containers & Computational Reproducibility
The bioinformatics reproducibility crisis, POSIX portability, monolithic bash scripts vs. declarative DAG workflows, and why pipeline orchestration engines matter.
2: Nextflow DSL2 Core Syntax, Channels & Reactive Flow
Nextflow DSL2 paradigms, Channel types (queue vs value), channel factories (fromPath, fromFilePairs), and operators (map, filter, combine, join, branch).
3: Processes, Directives & Script Blocks
Input, output, script, and when blocks, process directives (cpus, memory, time, publishDir, tag), exit codes, and modular process reuse.
4: Containerization with Docker, Singularity & Apptainer
Dockerfile construction for bioinformatics, multi-stage builds, rootless Singularity/Apptainer on HPC environments, and BioContainers registry.
5: Package Management with Conda, Mamba & Seqera Wave
Conda environment recipes, Mamba solver acceleration, conda-lock deterministic pinning, and real-time on-demand container synthesis with Seqera Wave.
6: Building an End-to-End RNA-Seq Pipeline in Nextflow DSL2
Complete, working Nextflow pipeline: Samplesheet parsing, FastQC, Fastp trimming, STAR splice-aware mapping, FeatureCounts quantification, and MultiQC aggregation.
7: nf-core Architecture, Linting & Community Modules
nf-core community guidelines, nf-core tools CLI, module creation, JSON schema parameter validation, and continuous integration testing.
8: Robust Error Handling, Retries & Dynamic Resource Allocation
The errorStrategy directive, dynamic memory/time scaling based on attempt counts, exit code handling, and cache invalidation mechanics.
9: HPC Cluster Integration: SLURM, SGE & LSF Executors
Connecting Nextflow to institutional HPC clusters, nextflow.config profile design, SLURM partition directives, queue limits, and scratch filesystems.
10: Cloud Execution: AWS Batch, S3 & AWS HealthOmics
Serverless cloud genomics, AWS Batch compute environments, S3 bucket staging, IAM roles, and AWS HealthOmics native workflow execution.
11: Cloud Execution: Google Cloud Batch & Azure CycleCloud
Google Cloud Batch, Cloud Storage (GCS) buckets, Service Accounts, and Microsoft Azure Batch / CycleCloud infrastructure.
12: Pipeline Monitoring & Observability with Seqera Platform (Tower)
Seqera Platform (Tower) observability, access tokens, real-time DAG visualizations, CPU/RAM utilization metrics, cost estimation, and team workspaces.
13: Snakemake Comparison, GitHub Actions CI/CD & Zenodo Archiving
Snakemake rules vs Nextflow processes, automated linting and testing with GitHub Actions CI, DOI minting on Zenodo, and pipeline publication best practices.
📚 Foundational Literature & Methodology Benchmarks
Workflow Engines & Nextflow
- Di Tommaso P, Chatzou M, Floden EW, et al. (2017). Nextflow enables reproducible computational workflows. Nat Biotechnol 35(4):316–319. doi:10.1038/nbt.3820
- Ewels PA, Peltzer A, Fillinger S, et al. (2020). The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol 38(3):276–278. doi:10.1038/s41587-020-0439-x
- Wratten L, Wilm A, Göke J. (2021). Reproducible, scalable, and shareable analysis workflows with bioinformatics pipeline frameworks. Nat Methods 18(10):1161–1168. doi:10.1038/s41592-021-01254-9
Containerization & Isolation
- Boettiger C. (2015). An introduction to Docker for reproducible research. ACM SIGOPS Oper Syst Rev 49(1):71–79. doi:10.1145/2723872.2723882
- Kurtzer GM, Sochat V, Bauer MW. (2017). Singularity: scientific containers for mobility of compute. PLOS ONE 12(5):e0177459. doi:10.1371/journal.pone.0177459
- Grüning B, Dale R, Sjödin A, et al. (2018). Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods 15(7):475–476. doi:10.1038/s41592-018-0046-7
Standards & Reproducibility
- Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3:160018. doi:10.1038/sdata.2016.18
- Mölder F, Jablonski KP, Letcher B, et al. (2021). Sustainable data analysis with Snakemake. F1000Res 10:33. doi:10.12688/f1000research.29032.2
- Ewels P, Magnusson M, Lundin S, Käller M. (2016). MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics 32(19):3047–3048. doi:10.1093/bioinformatics/btw354