omics.co.in logo omics.co.in

Day 1: Biological Databases & Programmatic Sequence Retrieval (NCBI, UniProt, PDB)

Tested Stacks: GATK 4.6.1 Samtools 1.21 Nextflow 24.10 Seurat v5.1 Python 3.12
Prerequisites: Basic Linux CLI & Sequencing Formats
📚 Chapter 1 of 15 0% Course Completed

1. Foundations of Biological Databases in Molecular Biology

Modern computational molecular biology rests upon centralized, standardized, and curated biological data repositories. High-throughput automated sequencing and structural biology consortia deposit gigabytes of sequence and coordinate information daily. To extract biological insight, researchers must understand the foundational taxonomy of biological repositories:

Database CategoryPrimary Representative RepositoriesData Characteristics & ProvenanceStandard Unique IdentifiersBiomedical Application
Primary Sequence DatabasesINSDC (NCBI GenBank, ENA, DDBJ)Archival, directly submitted nucleotide sequences from authors; no modifications without submitter approval. Highly redundant.GenBank accession (e.g. U00096.3), GI numbers (legacy)Raw patent sequences, novel isolate deposits, raw WGS submissions
Curated Non-Redundant DatabasesNCBI RefSeq (Reference Sequence)Expert-curated, computationally validated non-redundant reference genomes, transcripts, and proteins.NM_ (mRNA), NP_ (protein), NC_ (complete chromosome), NG_ (genomic region)Diagnostic variant calling, clinical transcript alignment, reference genome assemblies
Protein KnowledgebasesUniProtKB / Swiss-Prot & TrEMBLSwiss-Prot: manually annotated, peer-reviewed, functional domains, PTMs, catalytic residues.
TrEMBL: unreviewed automated translation of EMBL/GenBank CDS.
UniProt accession (e.g. P04637 for Human TP53, P01308 for Insulin)Drug target identification, functional domain mapping, enzymology mechanisms
Macromolecular Structure RepositorieswwPDB (RCSB PDB, PDBe, PDBj)Experimentally determined 3D atomic coordinates from X-ray crystallography, Cryo-EM, and Solution NMR.4-character alphanumeric PDB ID (e.g. 1TUP, 6VXX, 7KRR) or 8-character extendedStructure-based drug design, in silico docking, molecular dynamics simulations
Specialized Molecular Biology DatabasesEnsembl, KEGG, ClinVar, miRBaseSpecialized integration: gene models, metabolic wiring diagrams, clinical variant classifications, and non-coding RNAs.Ensembl Gene (ENSG...), KEGG Entry (hsa04115), ClinVar RCV (RCV...)Systems biology, precision oncology, pathway enrichment, microRNA target prediction

2. The FASTA Sequence Format Standard

The universal currency of sequence analysis is the FASTA format. A standard FASTA file consists of a single-line definition header starting with a greater-than symbol (>), followed immediately by a unique sequence identifier and arbitrary descriptive metadata, followed by lines of nucleotide or amino acid codes using IUPAC standard nomenclature:

>NM_000546.6 Homo sapiens tumor protein p53 (TP53), transcript variant 1, mRNA
GATGGGATTGGGGTTTTCCCCTCCCATGTGCTCAAGACTGGCGCTAAAAGTTTTGAGCTTCTCAAAAGTC
TAGTCCAAAAATAATTTCATCACTTTTTTTTTTTTTTTTTAATTTTTTTTTTTTTTTTTTTTTTTTTTTT
TTTTTTTTTTTTTTTGTCTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
...

IUPAC Degenerate Nucleotide & Amino Acid Codes

When sequencing primers, degeneracies, or unknown bases are present, computational tools rely on standard IUPAC ambiguous nucleotide codes:

SymbolBases RepresentedMnemonic / Biological MeaningComplement
RA or GpuRineY (C or T)
YC or TpYrimidineR (A or G)
MA or CaMino groupK (G or T)
KG or TKeto groupM (A or C)
SG or CStrong interaction (3 H-bonds)S (G or C)
WA or TWeak interaction (2 H-bonds)W (A or T)
NA, C, G, or TaNy base (unknown)N

3. Programmatic Sequence Retrieval with NCBI Entrez E-utilities

NCBI's Entrez E-utilities (Entrez Programming Utilities) represent a suite of server-side REST APIs providing programmatic access to all NCBI databases. The core utility pipeline involves:

  • esearch: Queries an NCBI database with Boolean logic and returns a list of matching Primary Identification (UID) numbers.
  • efetch: Downloads complete biological records (FASTA, GenBank XML, ASN.1) for specified UIDs.
  • esummary: Returns brief document summaries (DocSums) without downloading complete sequences.
  • elink: Traverses cross-database relationships (e.g. linking a gene ID in gene to its related structures in structure or articles in pubmed).
NCBI API Polling Etiquette

Always provide your email address (Entrez.email = "researcher@institution.edu") and an NCBI API Key (obtained from your NCBI account settings). Without an API key, NCBI limits requests to 3 queries per second; with an API key, the limit increases to 10 queries per second.

Production Biopython Script: Automated Multi-Database Sequence Fetcher

#!/usr/bin/env python3
"""
Day 1 Practical: Programmatic Retrieval of Genes, Transcripts, and Proteins
Integrates NCBI Entrez, UniProt REST, and RCSB PDB APIs.
"""

import sys
import json
import urllib.request
from Bio import Entrez, SeqIO

Entrez.email = "bioinformatics.researcher@omics.co.in"
Entrez.tool = "OmicsPortalResearch"

def fetch_ncbi_nucleotide(accession: str, rettype="fasta"):
    """Fetch nucleotide record from NCBI Nucleotide/RefSeq."""
    print(f"[NCBI] Retrieving {accession} (type: {rettype})...")
    try:
        handle = Entrez.efetch(db="nuccore", id=accession, rettype=rettype, retmode="text")
        record_text = handle.read()
        handle.close()
        return record_text
    except Exception as e:
        print(f"Error fetching NCBI record {accession}: {e}", file=sys.stderr)
        return None

def fetch_uniprot_entry(uniprot_id: str):
    """Retrieve curated Swiss-Prot annotations via UniProt REST API."""
    url = f"https://rest.uniprot.org/uniprotkb/{uniprot_id}.json"
    print(f"[UniProt] Querying curated JSON for {uniprot_id}...")
    req = urllib.request.Request(url, headers={"User-Agent": "BiopythonWorkflow/1.0"})
    with urllib.request.urlopen(req) as response:
        if response.status == 200:
            data = json.loads(response.read().decode())
            gene_name = data.get("genes", [{}])[0].get("geneName", {}).get("value", "N/A")
            organism = data.get("organism", {}).get("scientificName", "N/A")
            function = "N/A"
            for comment in data.get("comments", []):
                if comment.get("commentType") == "FUNCTION":
                    function = comment.get("texts", [{}])[0].get("value", "N/A")
                    break
            seq = data.get("sequence", {}).get("value", "")
            return {
                "accession": uniprot_id,
                "gene": gene_name,
                "organism": organism,
                "function": function,
                "length": len(seq),
                "sequence": seq
            }
    return None

def fetch_pdb_structure(pdb_id: str, out_file: str):
    """Download macromolecular 3D coordinate file from RCSB PDB."""
    url = f"https://files.rcsb.org/download/{pdb_id.upper()}.cif"
    print(f"[RCSB PDB] Downloading mmCIF coordinates for {pdb_id.upper()}...")
    urllib.request.urlretrieve(url, out_file)
    print(f"Saved {pdb_id.upper()} mmCIF coordinates to {out_file}.")

if __name__ == "__main__":
    # 1. Fetch Human TP53 RefSeq mRNA
    tp53_fasta = fetch_ncbi_nucleotide("NM_000546.6", rettype="fasta")
    with open("TP53_human_transcript.fasta", "w") as f:
        f.write(tp53_fasta)
    print("Saved TP53_human_transcript.fasta")

    # 2. Fetch Human Insulin from UniProt (P01308)
    insulin_data = fetch_uniprot_entry("P01308")
    print(f"Insulin Gene: {insulin_data['gene']}, Organism: {insulin_data['organism']}")
    print(f"Sequence Length: {insulin_data['length']} amino acids")
    print(f"First 30 AA: {insulin_data['sequence'][:30]}...")

    # 3. Download PDB 1TUP (p53 core domain bound to DNA)
    fetch_pdb_structure("1tup", "1tup.cif")

4. Hands-On Laboratory Exercise

  1. Query NCBI using esearch for the human BRCA1 gene (UID search: BRCA1[Gene Name] AND human[Organism] AND RefSeq[Filter]).
  2. Download the canonical mRNA transcript (NM_007294.4) and canonical protein (NP_009225.1) using the automated script.
  3. Inspect the FASTA header and verify the CDS coordinates in the corresponding GenBank flat file.
⚠️

Common Pitfalls & Troubleshooting Checklist

  • Java Heap Exhaustion (GATK / Picard): If processes terminate with java.lang.OutOfMemoryError, set your environment variable explicitly: export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp".
  • Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with samtools index -@ 4 sample.bam.
  • Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority: conda config --set channel_priority strict.
  • Chromosome Naming Contamination: Always verify whether reference FASTA headers use chr1 or 1; divergent naming causes silent alignment drops.
Was this guide helpful to your research?
Help us maintain 100% accurate, peer-reviewed bioinformatics standards.
📝 Propose Edit on GitHub ↗