Day 1: Biological Databases & Programmatic Sequence Retrieval (NCBI, UniProt, PDB)
1. Foundations of Biological Databases in Molecular Biology
Modern computational molecular biology rests upon centralized, standardized, and curated biological data repositories. High-throughput automated sequencing and structural biology consortia deposit gigabytes of sequence and coordinate information daily. To extract biological insight, researchers must understand the foundational taxonomy of biological repositories:
| Database Category | Primary Representative Repositories | Data Characteristics & Provenance | Standard Unique Identifiers | Biomedical Application |
|---|---|---|---|---|
| Primary Sequence Databases | INSDC (NCBI GenBank, ENA, DDBJ) | Archival, directly submitted nucleotide sequences from authors; no modifications without submitter approval. Highly redundant. | GenBank accession (e.g. U00096.3), GI numbers (legacy) | Raw patent sequences, novel isolate deposits, raw WGS submissions |
| Curated Non-Redundant Databases | NCBI RefSeq (Reference Sequence) | Expert-curated, computationally validated non-redundant reference genomes, transcripts, and proteins. | NM_ (mRNA), NP_ (protein), NC_ (complete chromosome), NG_ (genomic region) | Diagnostic variant calling, clinical transcript alignment, reference genome assemblies |
| Protein Knowledgebases | UniProtKB / Swiss-Prot & TrEMBL | Swiss-Prot: manually annotated, peer-reviewed, functional domains, PTMs, catalytic residues. TrEMBL: unreviewed automated translation of EMBL/GenBank CDS. | UniProt accession (e.g. P04637 for Human TP53, P01308 for Insulin) | Drug target identification, functional domain mapping, enzymology mechanisms |
| Macromolecular Structure Repositories | wwPDB (RCSB PDB, PDBe, PDBj) | Experimentally determined 3D atomic coordinates from X-ray crystallography, Cryo-EM, and Solution NMR. | 4-character alphanumeric PDB ID (e.g. 1TUP, 6VXX, 7KRR) or 8-character extended | Structure-based drug design, in silico docking, molecular dynamics simulations |
| Specialized Molecular Biology Databases | Ensembl, KEGG, ClinVar, miRBase | Specialized integration: gene models, metabolic wiring diagrams, clinical variant classifications, and non-coding RNAs. | Ensembl Gene (ENSG...), KEGG Entry (hsa04115), ClinVar RCV (RCV...) | Systems biology, precision oncology, pathway enrichment, microRNA target prediction |
2. The FASTA Sequence Format Standard
The universal currency of sequence analysis is the FASTA format. A standard FASTA file consists of a single-line definition header starting with a greater-than symbol (>), followed immediately by a unique sequence identifier and arbitrary descriptive metadata, followed by lines of nucleotide or amino acid codes using IUPAC standard nomenclature:
>NM_000546.6 Homo sapiens tumor protein p53 (TP53), transcript variant 1, mRNA
GATGGGATTGGGGTTTTCCCCTCCCATGTGCTCAAGACTGGCGCTAAAAGTTTTGAGCTTCTCAAAAGTC
TAGTCCAAAAATAATTTCATCACTTTTTTTTTTTTTTTTTAATTTTTTTTTTTTTTTTTTTTTTTTTTTT
TTTTTTTTTTTTTTTGTCTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
...IUPAC Degenerate Nucleotide & Amino Acid Codes
When sequencing primers, degeneracies, or unknown bases are present, computational tools rely on standard IUPAC ambiguous nucleotide codes:
| Symbol | Bases Represented | Mnemonic / Biological Meaning | Complement |
|---|---|---|---|
| R | A or G | puRine | Y (C or T) |
| Y | C or T | pYrimidine | R (A or G) |
| M | A or C | aMino group | K (G or T) |
| K | G or T | Keto group | M (A or C) |
| S | G or C | Strong interaction (3 H-bonds) | S (G or C) |
| W | A or T | Weak interaction (2 H-bonds) | W (A or T) |
| N | A, C, G, or T | aNy base (unknown) | N |
3. Programmatic Sequence Retrieval with NCBI Entrez E-utilities
NCBI's Entrez E-utilities (Entrez Programming Utilities) represent a suite of server-side REST APIs providing programmatic access to all NCBI databases. The core utility pipeline involves:
esearch: Queries an NCBI database with Boolean logic and returns a list of matching Primary Identification (UID) numbers.efetch: Downloads complete biological records (FASTA, GenBank XML, ASN.1) for specified UIDs.esummary: Returns brief document summaries (DocSums) without downloading complete sequences.elink: Traverses cross-database relationships (e.g. linking a gene ID ingeneto its related structures instructureor articles inpubmed).
Always provide your email address (Entrez.email = "researcher@institution.edu") and an NCBI API Key (obtained from your NCBI account settings). Without an API key, NCBI limits requests to 3 queries per second; with an API key, the limit increases to 10 queries per second.
Production Biopython Script: Automated Multi-Database Sequence Fetcher
#!/usr/bin/env python3
"""
Day 1 Practical: Programmatic Retrieval of Genes, Transcripts, and Proteins
Integrates NCBI Entrez, UniProt REST, and RCSB PDB APIs.
"""
import sys
import json
import urllib.request
from Bio import Entrez, SeqIO
Entrez.email = "bioinformatics.researcher@omics.co.in"
Entrez.tool = "OmicsPortalResearch"
def fetch_ncbi_nucleotide(accession: str, rettype="fasta"):
"""Fetch nucleotide record from NCBI Nucleotide/RefSeq."""
print(f"[NCBI] Retrieving {accession} (type: {rettype})...")
try:
handle = Entrez.efetch(db="nuccore", id=accession, rettype=rettype, retmode="text")
record_text = handle.read()
handle.close()
return record_text
except Exception as e:
print(f"Error fetching NCBI record {accession}: {e}", file=sys.stderr)
return None
def fetch_uniprot_entry(uniprot_id: str):
"""Retrieve curated Swiss-Prot annotations via UniProt REST API."""
url = f"https://rest.uniprot.org/uniprotkb/{uniprot_id}.json"
print(f"[UniProt] Querying curated JSON for {uniprot_id}...")
req = urllib.request.Request(url, headers={"User-Agent": "BiopythonWorkflow/1.0"})
with urllib.request.urlopen(req) as response:
if response.status == 200:
data = json.loads(response.read().decode())
gene_name = data.get("genes", [{}])[0].get("geneName", {}).get("value", "N/A")
organism = data.get("organism", {}).get("scientificName", "N/A")
function = "N/A"
for comment in data.get("comments", []):
if comment.get("commentType") == "FUNCTION":
function = comment.get("texts", [{}])[0].get("value", "N/A")
break
seq = data.get("sequence", {}).get("value", "")
return {
"accession": uniprot_id,
"gene": gene_name,
"organism": organism,
"function": function,
"length": len(seq),
"sequence": seq
}
return None
def fetch_pdb_structure(pdb_id: str, out_file: str):
"""Download macromolecular 3D coordinate file from RCSB PDB."""
url = f"https://files.rcsb.org/download/{pdb_id.upper()}.cif"
print(f"[RCSB PDB] Downloading mmCIF coordinates for {pdb_id.upper()}...")
urllib.request.urlretrieve(url, out_file)
print(f"Saved {pdb_id.upper()} mmCIF coordinates to {out_file}.")
if __name__ == "__main__":
# 1. Fetch Human TP53 RefSeq mRNA
tp53_fasta = fetch_ncbi_nucleotide("NM_000546.6", rettype="fasta")
with open("TP53_human_transcript.fasta", "w") as f:
f.write(tp53_fasta)
print("Saved TP53_human_transcript.fasta")
# 2. Fetch Human Insulin from UniProt (P01308)
insulin_data = fetch_uniprot_entry("P01308")
print(f"Insulin Gene: {insulin_data['gene']}, Organism: {insulin_data['organism']}")
print(f"Sequence Length: {insulin_data['length']} amino acids")
print(f"First 30 AA: {insulin_data['sequence'][:30]}...")
# 3. Download PDB 1TUP (p53 core domain bound to DNA)
fetch_pdb_structure("1tup", "1tup.cif")
4. Hands-On Laboratory Exercise
- Query NCBI using
esearchfor the human BRCA1 gene (UID search:BRCA1[Gene Name] AND human[Organism] AND RefSeq[Filter]). - Download the canonical mRNA transcript (
NM_007294.4) and canonical protein (NP_009225.1) using the automated script. - Inspect the FASTA header and verify the CDS coordinates in the corresponding GenBank flat file.
Common Pitfalls & Troubleshooting Checklist
- Java Heap Exhaustion (GATK / Picard): If processes terminate with
java.lang.OutOfMemoryError, set your environment variable explicitly:export _JAVA_OPTIONS="-Xmx16g -Djava.io.tmpdir=/tmp". - Index Timestamp Mismatch: If tools complain that a BAM index is missing or out of date, regenerate with
samtools index -@ 4 sample.bam. - Conda Channel Conflicts: Avoid ABI incompatibilities between bioconda and conda-forge by setting strict priority:
conda config --set channel_priority strict. - Chromosome Naming Contamination: Always verify whether reference FASTA headers use
chr1or1; divergent naming causes silent alignment drops.