Module 6: Long-Read Filtering & Preprocessing: Chopper, Filtlong & Downsampling
1. The Rationale for Long-Read Filtering
While long reads are prized for spanning repetitive genomic elements, short degraded fragments ($< 500\text{ bp}$) and low-accuracy reads ($\text{Q} < 10$) consume alignment memory and introduce spurious structural variant signals without contributing to assembly contiguity.
Filtering long-read sequencing data requires balancing three trade-offs:
- Minimum Length Cutoff: Eliminates short degradation products that bias coverage and cause alignment ambiguities in segmental duplications. Standard threshold: $\ge 1,000\text{ bp}$.
- Quality Threshold: Discards reads with ambiguous base calls. For ONT R10.4.2 SUP, filtering at $\text{Q} \ge 10$ or $\text{Q} \ge 12$ removes $>95\%$ of unresolvable chimeric reads.
- Head & Tail Trimming: Removes adapter sequences and transient pore block artifacts at the extremities of translocating molecules.
2. High-Performance Filtering with Chopper
chopper is a multithreaded Rust implementation that replaces legacy Python-based tools like NanoFilt, processing millions of reads per minute:
# Filter reads to retain only length >= 1000 bp and mean Q-score >= 12
zcat basecalled_reads.fastq.gz | \
chopper --minlength 1000 --minqual 12 --threads 8 | \
pigz -p 8 > filtered_reads.fastq.gz
# Inspect retention statistics
echo "Raw reads: $(zcat basecalled_reads.fastq.gz | wc -l | awk '{print $1/4}')"
echo "Filtered reads: $(zcat filtered_reads.fastq.gz | wc -l | awk '{print $1/4}')"3. Coverage Normalization with Rasusa
Excessive sequencing coverage ($> 80-100\times$) in long-read de novo assembly or structural variant calling dramatically increases computation time and can introduce erroneous heterozygous calls due to accumulated sequencing artifacts. rasusa downsamples reads randomly to an exact targeted coverage depth:
# Downsample filtered reads to exactly 30x coverage for a 3.1 Gb human genome
rasusa \
--input filtered_reads.fastq.gz \
--coverage 30 \
--genome-size 3.1Gb \
--output downsampled_30x.fastq.gzCommon Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.