Chapter 4: Clinical Metadata Handling, Patient Cohort Filtering & Pivot Table Exploration

1. The Critical Synchrony of Metadata and Count Matrices

In all differential expression frameworks (DESeq2, edgeR, limma-voom), the computational engine requires two coordinated data structures:

1. The Count Matrix (\(M\)): An integer matrix with \(G\) rows (genes) and \(S\) columns (patient samples).
2. The Sample Metadata (\(P\)): A dataframe with \(S\) rows (patient samples) and \(C\) columns (clinical attributes such as diagnosis, stage, treatment, batch, age, sex).

2. Metadata Wrangling, Re-leveling & Contingency Tables

Biomedical sample sheets often contain messy annotations, mixed casing, missing values (NA), and improper default factor leveling. By default, R assigns factor levels alphabetically. If your condition column contains "Control" and "Treated", R will set "Control" as the reference baseline. However, if your groups are "WildType" and "Knockout", R will pick "Knockout" as baseline alphabetically unless explicitly re-leveled!

Complete Data Wrangling & Pivot Table Workflow

library(dplyr)
library(tidyr)

# 1. Simulated Raw Clinical Metadata Table
clinical_raw <- data.frame(
    Sample_ID = paste0("SAMPLE_", sprintf("%02d", 1:12)),
    Condition = c("Treated", "Control", "Treated", "Control", "treated", "Control",
                  "Treated", "Control", "Treated", "Control", "Treated", "Control"),
    Tumor_Stage = c("Stage_III", "Normal", "Stage_IV", "Normal", "Stage_III", "Normal",
                    "Stage_II", "Normal", "Stage_IV", "Normal", "Stage_II", "Normal"),
    Patient_Sex = c("M", "F", "F", "M", "M", "F", "F", "M", "M", "F", "M", "F"),
    Extraction_Batch = c("Batch_1", "Batch_1", "Batch_1", "Batch_1", "Batch_2", "Batch_2",
                         "Batch_2", "Batch_2", "Batch_3", "Batch_3", "Batch_3", "Batch_3"),
    stringsAsFactors = FALSE
)

# 2. Clean and Standardize Clinical Annotations
clinical_clean <- clinical_raw %>%
    mutate(
        # Fix casing anomalies
        Condition = ifelse(tolower(Condition) == "treated", "Treated", "Control"),
        # Explicitly declare factor and set baseline reference level
        Condition = factor(Condition, levels = c("Control", "Treated")),
        Tumor_Stage = factor(Tumor_Stage),
        Patient_Sex = factor(Patient_Sex),
        Extraction_Batch = factor(Extraction_Batch)
    )

# Set Sample_ID as row names
rownames(clinical_clean) <- clinical_clean$Sample_ID

# 3. Generate Clinical Contingency Pivot Tables
cat("=== Cross-Tabulation: Condition vs. Extraction Batch ===\n")
print(table(clinical_clean$Condition, clinical_clean$Extraction_Batch))

cat("\n=== Cross-Tabulation: Tumor Stage vs. Patient Sex ===\n")
print(table(clinical_clean$Tumor_Stage, clinical_clean$Patient_Sex))

# 4. Generate Mock Count Matrix and Verify Identity
set.seed(42)
gene_ids <- paste0("ENSG00000", sprintf("%05d", 1:500))
mock_counts <- matrix(
    rpois(500 * 12, lambda = 100),
    nrow = 500,
    ncol = 12,
    dimnames = list(gene_ids, clinical_clean$Sample_ID)
)

# 5. Strict Assertion Check
stopifnot(all(colnames(mock_counts) == rownames(clinical_clean)))
cat("\n-> Assertion Passed: Count matrix columns and clinical metadata rows match perfectly!\n")