ESM-2 / ESMFold (Meta FAIR)

Tier: 1 — Essential | Category: Protein Language Model / Structure Prediction

ESM-2 is a family of protein language models from Meta FAIR trained on 250 million protein sequences spanning evolutionary diversity. ESMFold uses the ESM-2 representations to enable single-sequence, fast atomic-resolution protein structure prediction without requiring multiple sequence alignments (MSAs). Released in 2022–2023, ESM-2 and ESMFold together represent the most accessible path from protein sequence to 3D structure ^{1Evolutionary-scale prediction of atomic-level protein structure with a language model2023 · Science · DOI 10.1126/science.ade2574Open reference}.

Model Architecture

ESM-2 uses a transformer encoder trained with masked language modeling on UniRef90/UniRef50. The largest model (ESM-2 3B parameters) achieves competitive structure prediction accuracy when combined with the ESMFold folding head. Unlike AlphaFold2 and RoseTTAFold, ESMFold runs inference in seconds per sequence on a single GPU because it requires no MSA computation ^{1Evolutionary-scale prediction of atomic-level protein structure with a language model2023 · Science · DOI 10.1126/science.ade2574Open reference}.

Key capabilities:

• Structure prediction: Atomic-resolution structures from single sequences in ~1 second

• Variant effect prediction: Zero-shot log-likelihood scores for missense mutations ^{2Language models enable zero-shot prediction of the effects of mutations on protein function2021 · NeurIPS · DOI 10.1101/2021.07.09.450648Open reference}

• Fitness prediction: Unsupervised fitness landscape exploration ^{3Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences2021 · PNAS · DOI 10.1073/pnas.2016239118Open reference}

• Transfer learning: Fine-tune representations for downstream tasks such as variant pathogenicity ^{4Cross-protein transfer learning substantially improves disease variant prediction2023 · Genome Biology · DOI 10.1186/s13059-023-03024-6 · PMID 37752479Open reference}

Applications in Neurodegeneration Research

ESM-2’s zero-shot mutation scoring capability makes it particularly valuable for prioritizing disease-associated variants in neurodegeneration-relevant proteins. For tau (MAPT), TDP-43 (TARDBP), alpha-synuclein (SNCA), and FUS, ESM-2 can compute the log-likelihood ratio ∆LLR for each patient-observed variant — predicting which missense mutations disrupt protein fold stability or evolutionary conservation, independent of any structural model.

ESMFold enables rapid structure prediction of the hundreds of splice isoforms observed in neurodegeneration-relevant proteins, such as the six tau isoforms implicated in different tauopathies. Because ESMFold runs in seconds per sequence, it is tractable to generate structural models for all known coding variants in a gene of interest, enabling systematic proteogenomic analysis at population scale.

Tutorial: Scoring tau variants with ESM-2

import torch
from esm import pretrained

# Load ESM-2 650M (or 3B for higher accuracy)
model, alphabet = pretrained.esm2_t33_650M_UR50D()
model.eval()

# Tau P301L mutation scoring (MAPT isoform 2N4R)
from esm.inverse_folding.util import score_sequence
batch_converter = alphabet.get_batch_converter()

# Wild-type tau sequence fragment around P301
wt = "VQIVYKPVDLSKVTSKCGSLGNIHHKPGGG"
mt = "VQIVYKPVDLSKVTSKCGSLGNIHHKPGGG".replace("P", "L", 1)  # P301L

data = [("wt", wt), ("P301L", mt)]
batch_labels, batch_strs, batch_tokens = batch_converter(data)

with torch.no_grad():
    results = model(batch_tokens, repr_layers=[33], return_contacts=False)
    logits = results["logits"]

# Compute per-residue log-likelihood difference
import torch.nn.functional as F
log_probs = F.log_softmax(logits, dim=-1)
# delta_LLR > 0 = mutation tolerated; < 0 = deleterious

Relevance to SciDEX

ESM-2/ESMFold enables two concrete applications within the Forge tool layer:

1. Variant triage: Score coding variants in neurodegeneration GWAS hits for functional effect, feeding prioritized candidates into Exchange hypotheses.

2. Fast structural context: Generate structural models for proteins with no experimental structure to inform Agora debate about mechanism.

The esm skill in SciDEX’s Forge layer wraps ESM-2 inference for variant effect scoring. Cross-reference hypothesis [[h-297a9dcbfa]] (FUS mutations alter stress granule material properties) where ESM-2 scores were used to assess pathogenicity of FUS missense variants.

Limitations

• ESMFold is less accurate than AlphaFold2 for multi-domain proteins and proteins with known homologues (MSA-based methods have an advantage)

• Zero-shot variant scoring is correlative, not causal: it captures evolutionary fitness, not disease causality

• No support for multi-chain complexes, ligands, or post-translational modifications (cf. AlphaFold3)

• Accuracy degrades for intrinsically disordered regions common in neurodegeneration proteins (tau, alpha-synuclein)

Cross-References

• [[ai-tool-esm3]] — ESM-3, the successor multimodal model

• [[technologies-esm-protein-language-model]] — Overview of ESM family

• [[technologies-alphafold]] — AlphaFold2 for MSA-based structure prediction

• [[ai-tool-alphafold3]] — AlphaFold3 for multi-molecular complexes

• [[mechanisms-evolutionary-scale-modeling-esm]] — Mechanistic context