omics.co.in logo omics.co.in
← Back to Directory Hub (All 18 Disciplines)
← Prev: Stage 16 | Next: Stage 18 →
Discipline 17 of 18 25 Essential Databases Stage 17: Foundation Archives & Benchmark Knowledgebases

📚 17. Databases & Reference Resources Used in NGS Studies

Curated international repositories of non-redundant reference genomes, clinical variant pathogenicity catalogs, population allele frequencies, and biological pathways.

Essential Software: NCBI RefSeq Ensembl UCSC UniProt gnomAD ClinVar
Comparative Analysis Production Benchmarks

Algorithm Architecture & Decision Matrices

Detailed comparison cards highlighting algorithmic trade-offs, strengths, weaknesses, and recommended production setups.

These curated reference archives, biological knowledgebases, and international nucleotide data centers are foundational components of next-generation sequencing pipelines — providing assembly references, variant effect benchmarks, taxonomic marker trees, and functional annotations.

Reference Genomes & Assemblies
NCBI RefSeq GenBank Ensembl UCSC Genome Browser
Curated non-redundant reference genomes (GRCh38 / T2T-CHM13), gene annotations, ENCODE tracks, and comparative vertebrate synteny.
Human Genetic Variation & Cancer
dbSNP ClinVar gnomAD COSMIC
Standard rsID variant catalogs, clinical evidence classifications (ACMG/AMP), population allele frequencies (140k+ WGS/WES), and somatic cancer mutations.
Microbiome, Phylogenetics & rRNA
GTDB SILVA Greengenes RDP
Standardized prokaryotic genome taxonomy (MAGs), quality-aligned 16S/18S and 23S/28S rRNA alignments, and naive Bayesian classifiers.
Proteins, Domains & Pathway Wiring
UniProt Pfam InterPro KEGG Gene Ontology (GO) STRING
Swiss-Prot manually curated proteins, protein domain HMMs, metabolic pathway wiring (KO), controlled vocabularies, and interaction networks.
Carbohydrate Enzymes, Resistome & AMR
CARD ResFinder dbCAN CAZy
Antibiotic resistance ontology (ARO), acquired antimicrobial resistance genes, carbohydrate-active enzymes, and glycoside hydrolase families.
Primary Raw Sequencing Archives (INSDC)
NCBI GEO NCBI SRA EBI ENA DDBJ
International high-throughput sequencing data hubs archiving open-access raw FASTQ/BAM files, expression matrices, and metadata.
Hardware & Execution Profiles

Primary Software & Algorithms in Biological Reference Databases

Direct command-line recipes, Conda environment specifications, memory footprints, and license metadata.

Protein Sequence & Function
🌐 Cloud / API REST API • SPARQL • Flatfiles

UniProt (Swiss-Prot / TrEMBL)

Docs ↗ By UniProt Consortium (EBI, SIB, PIR)
(License: CC-BY 4.0)

The world's premier protein resource. Swiss-Prot provides manually curated, peer-reviewed expert functional annotations; TrEMBL provides computationally analyzed translations.

Formats: Access: Web Portal, REST API, TSV, XML, FASTA, JSON
🌐 Public API & Cloud Downloads
curl -s "https://rest.uniprot.org/uniprotkb/P04637.json"
Vertebrate Genome Annotation
🌐 Cloud / API REST API • MySQL • BioMart

Ensembl Genome Browser

Docs ↗ By EMBL-EBI & Wellcome Sanger
(License: Apache-2.0)

Comprehensive genome information system for vertebrate and model organism genomes. Provides high-quality GENCODE gene models, regulatory features, variation databases, and REST endpoints.

Formats: Access: REST API, BioMart, GTF/GFF3, FASTA
🌐 High-availability REST API
curl -s "https://rest.ensembl.org/lookup/symbol/homo_sapiens/BRCA1"
Sequence Read Archive
🌐 Cloud / API Cloud Object Stores • SRA Toolkit

NCBI SRA / Entrez

Docs ↗ By National Center for Biotechnology Information
(License: Public Domain)

The primary public archive for high-throughput sequencing data, hosting petabytes of raw NGS reads from global biomedical and environmental sequencing projects.

Formats: Access: fastq-dump, SRA Toolkit, AWS S3 / GCP buckets
💻 SRA Toolkit client • High network bandwidth
conda install -c bioconda sra-tools
3D Macromolecular Structures
🌐 Cloud / API GraphQL • REST API • mmCIF

RCSB Protein Data Bank (PDB)

Docs ↗ By wwPDB Consortium
(License: CC0 1.0 Universal)

The single worldwide archive of experimentally determined 3D structures of biological macromolecules (X-ray crystallography, Cryo-EM, NMR). Houses >220,000 atomic coordinate models.

Formats: Access: REST, GraphQL, mmCIF (.cif), PDB
🌐 REST API & Global FTP
curl -s "https://data.rcsb.org/rest/v1/core/entry/4HHB"
Human Genetic Variation
🌐 Cloud / API GraphQL • Google BigQuery • Hail

gnomAD

Docs ↗ By Broad Institute & gnomAD Consortium
(License: ODbL v1.0)

Genome Aggregation Database spanning >800,000 diverse exomes and genomes to calculate human population allele frequencies, loss-of-function intolerance (pLI, LOEUF), and structural variants.

Formats: Access: Web GUI, GraphQL, Multi-sample VCF, Hail tables
🌐 GraphQL API / Google Cloud Bucket Downloads
curl -X POST -H 'Content-Type: application/json' -d '{"query":"{ gene(gene_symbol: \"BRCA1\") { gene_id } }"}' https://gnomad.broadinstitute.org/api
Genomic Variation & Human Health
🌐 Cloud / API FTP VCF • XML • E-utilities

ClinVar

Docs ↗ By NCBI / National Library of Medicine
(License: Public Domain)

Publicly accessible archive of reports of relationships between human genetic variations and phenotypes, with supporting evidence, ACMG/AMP star classifications, and clinical significance.

Formats: Access: Monthly VCF Releases, XML, Tab-delimited files
💻 Local VCF analysis or Web Portal
wget https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/clinvar.vcf.gz
Human Small Molecule Knowledgebase
🌐 Cloud / API XML • SDF • Web Portal

HMDB (Human Metabolome Database)

Docs ↗ By David Wishart Lab (Univ. of Alberta)
(License: Open Access (Academic))

Comprehensive open-access database detailing >220,000 small molecule metabolites found in human tissues, biofluids, and gut microbiomes, linked to chemical structures and disease pathways.

Formats: Access: Web Search, XML downloads, SDF structures
🌐 Web Portal & Full XML Archive
curl -s "https://hmdb.ca/metabolites/HMDB0000001.xml" | head -n 20
Proteomics Mass Spec Repository
🌐 Cloud / API REST API • Aspera / FTP

PRIDE & ProteomeXchange

Docs ↗ By Juan Antonio Vizcaíno Lab (EMBL-EBI)
(License: CC-BY 4.0)

The PRIDE PRoteomics IDEntifications database is the world's leading repository for mass spectrometry-based proteomics data submissions, peptide/protein identifications, and raw MS files.

Formats: Access: REST API v2, Aspera high-speed download, FTP
🌐 REST API & Global FTP
curl -s "https://www.ebi.ac.uk/pride/ws/archive/v2/projects/PXD000001"
Genomics Binary Toolkit
💻 < 8GB RAM C • B-Trees & BigWig/BigBed

UCSC Genome Utilities

Docs ↗ By Jim Kent & UCSC Genome Browser Team
(License: Free Academic / Open Source)

Essential suite of 100+ command-line tools for high-throughput genomics: wigToBigWig, bedToBigBed, liftOver (coordinate translation across genome assemblies), and twoBitToFa.

Formats: Input: WIG, BED, Chain, 2bit • Output: BigWig, BigBed, FASTA
💻 2-4 CPU Cores • 4 GB RAM
conda install -c bioconda ucsc-liftover ucsc-wigtobigwig
Recommended Workflow Progression

Proceed to Stage 18: Lab Starter Stack & Foundational Shortlist

Sequencing pipelines proceed sequentially. Move forward to the next analytical domain to explore downstream algorithms and execution profiles.

Advance to Stage 18 →
← Stage 16: Specialized Applications & Multi-Omics Directory Hub Stage 18: Foundational Lab Shortlist →