omics.co.in logo omics.co.in

1: Biological Databases, Data Schemas & Accession Ontologies

Tested Stacks: GATK 4.6.1 Samtools 1.21 Nextflow 24.10 Seurat v5.1 Python 3.12
Prerequisites: Basic Linux CLI & Sequencing Formats
📚 Chapter 1 of 13 0% Course Completed

Chapter 1: Biological Databases, Data Schemas & Accession Ontologies

Modern bioinformatics is founded upon structured, publicly accessible repositories of biological sequence, structure, and functional data. Navigating these databases and parsing their underlying schemas programmatically is the indispensable initial capability required of every computational biologist.

Curated vs. Uncurated Repositories: The UniProt Example

The Universal Protein Resource (UniProt) is divided into two distinct repositories:

  • UniProtKB/Swiss-Prot: Manually annotated and reviewed by expert curators who evaluate experimental evidence from peer-reviewed literature. Contains ~570,000 highly accurate entries with verified active sites, post-translational modifications, and domain boundaries.
  • UniProtKB/TrEMBL: Automatically translated from EMBL/GenBank/DDBJ nucleotide coding sequences and annotated using automated computational pipelines. Contains over 250 million entries, reflecting the exponential pace of whole-genome sequencing.

1. The Global Biological Database Ecosystem

  • NCBI Entrez (National Center for Biotechnology Information): Federates PubMed, Nucleotide, Protein, Gene, ClinVar, and dbSNP through a unified text query engine.
  • Ensembl & GENCODE: Provides high-quality, comprehensive reference gene annotations for vertebrates, rigorously defining canonical transcripts, exons, and splice variants.
  • Protein Data Bank (PDB): The worldwide single archive of experimentally determined 3D structures of biological macromolecules determined via X-ray crystallography, NMR, and cryo-EM.

2. Gene Ontology: Directed Acyclic Graphs (DAGs)

Free-text gene descriptions make automated computation impossible. The Gene Ontology (GO) provides a controlled, standardized vocabulary organized as a Directed Acyclic Graph across three orthologous domains:

  • Molecular Function (MF): Elemental biochemical activities of a gene product at the molecular level (e.g., ATP-dependent kinase activity, GO:0016301).
  • Biological Process (BP): Coordinated series of molecular events or biological pathways (e.g., glycolytic process, GO:0006096).
  • Cellular Component (CC): Subcellular anatomical structures and macromolecular complexes (e.g., mitochondrial inner membrane, GO:0005743).

3. Programmatic Database Retrieval with Python

# Query NCBI Nucleotide programmatically via Entrez E-utilities
import urllib.request
import xml.etree.ElementTree as ET

# 1. Search for human BRCA1 mRNA transcripts
search_url = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=nuccore&term=BRCA1[Gene]+AND+Homo+sapiens[Organism]+AND+refseq[filter]&retmode=json"
with urllib.request.urlopen(search_url) as resp:
    import json
    search_data = json.loads(resp.read().decode())
    uid_list = search_data['esearchresult']['idlist']
    print(f"Discovered {len(uid_list)} RefSeq UIDs: {uid_list[:3]}")

# 2. Fetch the FASTA record for the primary hit
fetch_url = f"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&id={uid_list[0]}&rettype=fasta&retmode=text"
with urllib.request.urlopen(fetch_url) as resp:
    fasta_text = resp.read().decode()
    print("FASTA Header:", fasta_text.splitlines()[0])
⚠️

Common Pitfalls & Troubleshooting Checklist

  • Java Heap Exhaustion (GATK / Picard): If processes terminate with java.lang.OutOfMemoryError, set your environment variable explicitly: export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp".
  • Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with samtools index -@ 4 sample.bam.
  • Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority: conda config --set channel_priority strict.
  • Chromosome Naming Contamination: Always verify whether reference FASTA headers use chr1 or 1; divergent naming causes silent alignment drops.
Was this guide helpful to your research?
Help us maintain 100% accurate, peer-reviewed bioinformatics standards.
📝 Propose Edit on GitHub ↗