omics.co.in logo omics.co.in

Duplicate Marking, ENCODE Blacklist Masking & Cross-Correlation Metrics (NSC/RSC)

Tested Stacks: GATK 4.6.1 Samtools 1.21 Nextflow 24.10 Seurat v5.1 Python 3.12
Prerequisites: Basic Linux CLI & Sequencing Formats
📚 Chapter 4 of 10 0% Course Completed
PHASE 2 • DUPLICATES & BLACKLISTS Track 24 • 10 Modular Chapters Huntington's Disease H3K4me3 Case Study

Duplicate Marking, ENCODE Blacklist Masking & Cross-Correlation Metrics (NSC/RSC)

Differentiating optical from PCR duplicates with Sambamba / Picard, masking high-signal anomalous regions using ENCODE v2/v3 Unified Blacklists with bedtools, and computing Normalized Strand Cross-correlation (NSC) and Relative Strand Cross-correlation (RSC) with phantompeakqualtools.

⏱️ Estimated Study: 50–60 mins 🧪 Workflows: FastQC, Bowtie2, SAMtools, MACS3, DiffBind, ChIPseeker, deepTools, IGV 📋 Level: Hands-On Epigenomics • Real HD Post-Mortem Data

1. Duplicate Marking: Sambamba & Picard

Because ChIP-seq yields nanogram quantities of immunoprecipitated DNA, extensive PCR cycles during library preparation can generate significant PCR duplication bias. While natural chromatin fragmentation can produce overlapping reads at hyper-active promoter summits, excessive duplication distorts peak heights. We use sambamba markdup to identify and flag duplicates:

for sample in HD_Rep1 HD_Rep2 Control_Rep1 Control_Rep2 HD_Input Control_Input; do
    echo "Marking PCR duplicates for ${sample}..."
    sambamba markdup -t 6 -r \
        03_filtered/${sample}.q30.bam \
        03_filtered/${sample}.rmdup.bam
    
    samtools index 03_filtered/${sample}.rmdup.bam
done

2. Masking ENCODE Unified Blacklists with bedtools

The ENCODE project has identified anomalous, high-signal regions across the human genome (centromeric repeats, ribosomal DNA, satellite DNA) that non-specifically pull down fragments regardless of antibody specificity. Leaving blacklisted regions unmasked results in false-positive peak calls.

# Download ENCODE v2 GRCh38 Unified Blacklist BED
wget -qO hg38-blacklist.v2.bed.gz "https://github.com/Boyle-Lab/Blacklist/raw/master/lists/hg38-blacklist.v2.bed.gz"
gunzip hg38-blacklist.v2.bed.gz

# Filter out reads overlapping blacklisted coordinates
for sample in HD_Rep1 HD_Rep2 Control_Rep1 Control_Rep2 HD_Input Control_Input; do
    bedtools intersect -v -abam 03_filtered/${sample}.rmdup.bam \
        -b hg38-blacklist.v2.bed > 03_filtered/${sample}.clean.bam
    
    samtools index 03_filtered/${sample}.clean.bam
done

3. Strand Cross-Correlation Metrics: NSC & RSC

Cross-correlation analysis evaluates ChIP enrichment independently of peak calling by assessing the clustering of reads on opposing strands (+ and -) separated by the average sonicated fragment length ($d$). We run phantompeakqualtools (run_spp.R):

  • Normalized Strand Cross-correlation (NSC): Ratio of the maximum cross-correlation peak height to the background. Acceptable: NSC ≥ 1.05. High quality: NSC ≥ 1.10.
  • Relative Strand Cross-correlation (RSC): Ratio of fragment-length peak height to read-length phantom peak height. Acceptable: RSC ≥ 0.8. High quality: RSC ≥ 1.0.
Rscript /opt/phantompeakqualtools/run_spp.R \
    -c=03_filtered/HD_Rep1.clean.bam \
    -savp=06_qc_reports/HD_Rep1_crosscorr.pdf \
    -out=06_qc_reports/HD_Rep1_crosscorr.txt
⚠️

Common Pitfalls & Troubleshooting Checklist

  • Java Heap Exhaustion (GATK / Picard): If processes terminate with java.lang.OutOfMemoryError, set your environment variable explicitly: export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp".
  • Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with samtools index -@ 4 sample.bam.
  • Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority: conda config --set channel_priority strict.
  • Chromosome Naming Contamination: Always verify whether reference FASTA headers use chr1 or 1; divergent naming causes silent alignment drops.
Was this guide helpful to your research?
Help us maintain 100% accurate, peer-reviewed bioinformatics standards.
📝 Propose Edit on GitHub ↗