Duplicate Marking, ENCODE Blacklist Masking & Cross-Correlation Metrics (NSC/RSC)
Duplicate Marking, ENCODE Blacklist Masking & Cross-Correlation Metrics (NSC/RSC)
Differentiating optical from PCR duplicates with Sambamba / Picard, masking high-signal anomalous regions using ENCODE v2/v3 Unified Blacklists with bedtools, and computing Normalized Strand Cross-correlation (NSC) and Relative Strand Cross-correlation (RSC) with phantompeakqualtools.
1. Duplicate Marking: Sambamba & Picard
Because ChIP-seq yields nanogram quantities of immunoprecipitated DNA, extensive PCR cycles during library preparation can generate significant PCR duplication bias. While natural chromatin fragmentation can produce overlapping reads at hyper-active promoter summits, excessive duplication distorts peak heights. We use sambamba markdup to identify and flag duplicates:
for sample in HD_Rep1 HD_Rep2 Control_Rep1 Control_Rep2 HD_Input Control_Input; do
echo "Marking PCR duplicates for ${sample}..."
sambamba markdup -t 6 -r \
03_filtered/${sample}.q30.bam \
03_filtered/${sample}.rmdup.bam
samtools index 03_filtered/${sample}.rmdup.bam
done2. Masking ENCODE Unified Blacklists with bedtools
The ENCODE project has identified anomalous, high-signal regions across the human genome (centromeric repeats, ribosomal DNA, satellite DNA) that non-specifically pull down fragments regardless of antibody specificity. Leaving blacklisted regions unmasked results in false-positive peak calls.
# Download ENCODE v2 GRCh38 Unified Blacklist BED
wget -qO hg38-blacklist.v2.bed.gz "https://github.com/Boyle-Lab/Blacklist/raw/master/lists/hg38-blacklist.v2.bed.gz"
gunzip hg38-blacklist.v2.bed.gz
# Filter out reads overlapping blacklisted coordinates
for sample in HD_Rep1 HD_Rep2 Control_Rep1 Control_Rep2 HD_Input Control_Input; do
bedtools intersect -v -abam 03_filtered/${sample}.rmdup.bam \
-b hg38-blacklist.v2.bed > 03_filtered/${sample}.clean.bam
samtools index 03_filtered/${sample}.clean.bam
done3. Strand Cross-Correlation Metrics: NSC & RSC
Cross-correlation analysis evaluates ChIP enrichment independently of peak calling by assessing the clustering of reads on opposing strands (+ and -) separated by the average sonicated fragment length ($d$). We run phantompeakqualtools (run_spp.R):
- Normalized Strand Cross-correlation (NSC): Ratio of the maximum cross-correlation peak height to the background. Acceptable: NSC ≥ 1.05. High quality: NSC ≥ 1.10.
- Relative Strand Cross-correlation (RSC): Ratio of fragment-length peak height to read-length phantom peak height. Acceptable: RSC ≥ 0.8. High quality: RSC ≥ 1.0.
Rscript /opt/phantompeakqualtools/run_spp.R \
-c=03_filtered/HD_Rep1.clean.bam \
-savp=06_qc_reports/HD_Rep1_crosscorr.pdf \
-out=06_qc_reports/HD_Rep1_crosscorr.txtCommon Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.