Chapter 2: Nextflow DSL2 Core Syntax, Channels & Reactive Flow

Chapter 2: Nextflow DSL2 Core Syntax, Channels & Reactive Flow

Nextflow DSL2 (Domain Specific Language 2) enforces clean modularity, separating process definitions from workflow logic. At the heart of Nextflow lies the concept of Channels: asynchronous streams that transport data items between processes.

1. Core Channel Factories for Bioinformatics

// 1. Reading single files into a queue channel
ch_reads = Channel.fromPath( 'data/*.fastq.gz' )

// 2. Reading Paired-End FASTQ files with automatic regex grouping
// Emits: [ sample_id, [ read1.fastq.gz, read2.fastq.gz ] ]
ch_paired_reads = Channel.fromFilePairs( 'data/*_{1,2}.fastq.gz' )

// 3. Creating a broadcast Value Channel for reference genome
ch_genome = Channel.value( file('/ref/GRCh38.fasta') )

// 4. Emitting a channel from a sample sheet CSV
ch_samplesheet = Channel.fromPath('samplesheet.csv')
    | splitCsv(header: true)
    | map { row -> tuple(row.sample_id, file(row.fastq_1), file(row.fastq_2)) }

2. Essential Channel Operators

Channel operators transform, filter, and combine data streams using pipe syntax (|).

// Operator 1: map (Transform elements)
ch_paired_reads
    | map { meta, fastqs -> tuple( meta, fastqs[0] ) } // Extract only R1
    | view { "Sample: ${it[0]}, R1: ${it[1]}" }

// Operator 2: filter (Select elements matching condition)
ch_samplesheet
    | filter { sample_id, r1, r2 -> sample_id.startsWith('TUMOR') }

// Operator 3: combine (Cartesian product of channels)
// Combines every sample with every bed interval
ch_samples.combine( ch_intervals )

// Operator 4: join (Inner join based on matching key)
// Joins BAM channel and VCF channel having the same sample_id
ch_bams.join( ch_vcfs )

3. The Branch Operator: Splitting Data Streams

The branch operator directs channel elements into distinct output sub-channels based on logical criteria (e.g. single-end vs paired-end sequencing, or control vs tumor samples).

ch_reads.branch {
    single_end: it[1] instanceof Path
    paired_end: it[1] instanceof List
}.set { ch_branched }

ch_branched.single_end.view { "Single-end: $it" }
ch_branched.paired_end.view { "Paired-end: $it" }