Chapter 4: The AlphaFold Revolution: Evoformer & Invariant Point Attention
Chapter 4: The AlphaFold Revolution: Evoformer & Invariant Point Attention
In CASP14 (2020), DeepMind's AlphaFold2 solved the 50-year-old protein folding problem, demonstrating atomic accuracy comparable to X-ray crystallography across diverse protein folds. Unlike classical physics simulations, AlphaFold directly learns the mapping from evolutionary sequence co-variation to 3D spatial coordinates.
The Two Evolutionary Representations
AlphaFold continuously exchanges information between two core mathematical matrices:
- MSA Representation: Size \(N_{\text{seq}} \times N_{\text{res}}\). Captures co-evolution: if residue 45 mutates from positive to negative, residue 120 mutates from negative to positive to preserve a salt bridge.
- Pair Representation: Size \(N_{\text{res}} \times N_{\text{res}}\). Directly models pairwise spatial distances and orientations between all residue pairs.
1. The Evoformer Block Architecture
The Evoformer processes the MSA and Pair representations through 48 stacked transformer blocks:
- Row-wise & Column-wise Gated Self-Attention: Attends across sequences and across residues in the MSA.
- Outer Product Mean: Computes the outer product of MSA columns to continuously inject evolutionary co-variation directly into the Pair representation.
- Triangular Self-Attention (Triangle Multiplicative Update): Enforces the triangle inequality in Euclidean space (\(d_{ik} \le d_{ij} + d_{jk}\)). If residue A is near B, and B is near C, A and C must be constrained.
2. The 3D Structure Module & Invariant Point Attention (IPA)
The Structure Module directly generates 3D atomic coordinates without template guidance:
- It treats every residue as an independent rigid body reference frame ("residue gas") initialized at the origin (\(x=0, y=0, z=0\)).
- Invariant Point Attention (IPA): A specialized geometric attention mechanism that is invariant under 3D global translations and rotations \(SE(3)\).
- Iteratively updates the position vector \(\vec{x}\) and orientation quaternion \(R\) of each residue frame until the complete protein fold coalesces.