Biological benchmarks

82 benchmark records in release 2026-10-12-fcce59cc481b (gzip).

Broad tasks, specific protocols and evaluators have their own records. Follow each benchmark to its procedures and results.

• AlphaFold and ESMFold benchmark on PDB structures released 2022 to 2024

  Protein structure · Proteins and complexes

  Independent benchmark of AlphaFold2, AlphaFold3 and ESMFold on non-redundant PDB entries released between 2022 and 2024 whose sequences have no close homologue among pre-2022 structures.

• ATOM3D

  Molecular interactions

  ATOM3D provides molecular-structure datasets and utilities for task-specific evaluation.

• BEACON

  RNA and transcriptomes

  BEACON compares RNA representations across structural and functional downstream tasks.

• BEELINE

  Biological networks

  BEELINE compares inferred gene-regulatory edge rankings with reference networks.

• BEND

  DNA and genomes

  BEND evaluates DNA representations using task-specific genomic annotations and explicit split membership.

• Benet-Pages 2025 BRCA1/BRCA2 VUS reclassification study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• CAFA

  Protein function

  CAFA evaluates prospective protein-function predictions against annotations that become available after prediction submission.

• CAMI

  Microbes and communities

  CAMI is a community benchmark program for metagenomic computational methods.

• Cancer Immunotherapy Data Science Challenge

  Source study/challenge grouping of the exact imported protocols. No claim that this intake implements or reproduces an executable benchmark.

• CAPRI

  Protein structure

  CAPRI assesses blind predictions of protein-complex structures supplied before experimental publication.

• CASP

  Protein structure

  CASP assesses structure-prediction methods through blind predictions and category-specific evaluation.

• CIViC MCP 2026 evidence-retrieval evaluation

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• ClusPro Protein Docking Benchmark 5.0 study

  Proteins and complexes

  2020 author-reported study on 227 DockQ-evaluable BM5 targets; distinct from CAPRI61 submissions and modern cofolding benchmarks.

• DART-Eval

  DNA and genomes

  DART-Eval measures human regulatory-DNA representations under zero-shot, probing and fine-tuning regimes.

• Davis and KIBA drug-target affinity benchmark (DeepDTA split)

  Molecular interactions

  Drug-target affinity regression on Davis and KIBA with the predefined DeepDTA training and test folds (5:1).

• DeepProtein benchmark

  Protein function

  Benchmark of sequence-based, graph-based and pretrained protein models, plus two chemistry LLMs, over protein function, localization, interaction, epitope and paratope, developability, CRISPR repair and structure prediction tasks, with datasets taken from TAPE, PEER and TDC.

• Differential expression classification benchmark for perturbation models (Sun et al. 2026)

  Single cell

  Three-class (non-DE, up, down) differential expression prediction for unseen genetic perturbations within a cell line and for seen perturbations in a held-out cell line, on K562, RPE1, HepG2 and Jurkat essential-gene Perturb-seq screens.

• DNALONGBENCH

  DNA and genomes

  Five long-range DNA prediction tasks with inputs up to about 1 Mb: enhancer-target gene, eQTL, 3D contact map, regulatory sequence activity and transcription initiation signal prediction.

• ENIGMA 2024 BRCA evidence-calibration and curation study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• ESMFold2 Runs N’ Poses comparison

  Proteins and complexes

  Bounded complete Figure 2C Runs N’ Poses subpanel: all eleven printed labels, separated into single-sequence and MSA comparisons.

• FLIP

  Protein function

  FLIP evaluates protein-sequence representations using multiple deliberately defined train/test splits.

• FLIP2

  Protein function

  FLIP2 evaluates protein fitness prediction under deliberately shifted training and test distributions.

• FoldBench

  Source study/challenge grouping of the exact imported protocols. No claim that this intake implements or reproduces an executable benchmark.

• FramePool translation-prediction study

  RNA and transcriptomes

  2021 MPRA mean-ribosome-load comparisons for random and truncated-human UTR libraries across fixed and variable lengths.

• GenBench

  DNA and genomes

  Benchmark suite of short- and long-range genomic tasks for fine-tuned genomic foundation models.

• Gene-MTEB

  DNA and genomes

  Genomic embedding evaluations covering sequence classification and clustering. Accuracy and V-measure remain separate metrics and each source task retains its own comparison.

• GENEB

  DNA and genomes

  GENEB compares frozen genomic representations across classification tasks and label-budget regimes.

• GENEB leaderboard

  DNA and genomes

  Frozen-embedding probing benchmark of genomic foundation models over 100 tasks in 13 categories, with full-data, 10-shot and 1-shot regimes.

• Genie 3 unconditional short monomer comparison

  Proteins and complexes

  Bounded complete Table 3 comparison across 13 method/version rows. Five directional metrics are included; secondary-structure composition is retained separately without ranking.

• Genomic Benchmarks

  DNA and genomes

  Genomic Benchmarks packages genomic sequence-classification datasets with explicit versions and supplied train/test folders.

• GFMBench-API

  DNA and genomes

  Python interface that wraps existing genomic tasks (GUE, VariantBenchmarks, LRB, BEND, TraitGym, SongLab ClinVar, BRCA1, VEP-eval, LOL-EVE) and a new ClinVar indel task behind one evaluation API.

• GlycanML

  Glycans

  GlycanML evaluates glycan learning across multiple classification and interaction tasks.

• GUANinE v1.1

  DNA and genomes

  Genome Understanding and ANnotation in silico Evaluation, version 1.1: functional genomics, conservation, gene-expression and variant tasks on hg38.

• GUE

  DNA and genomes

  GUE evaluates genome understanding across multiple datasets, task types and species.

• Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)

  Single cell · Spatial omics

  Harmonised evaluation of six single-cell and spatial foundation models (Nicheformer, CellPLM, scGPT-spatial, GenePT-w, scELMo, Novae) on zero-shot and continually pretrained clustering, supervised cell type annotation, marker-gene concordance and perturbation prediction.

• HECTOR 2026 preprint: ENIGMA Evidence Repository validation

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocol. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation. Preprint; not peer reviewed.

• HEST-Benchmark

  Spatial omics

  HEST-Benchmark tests prediction of gene expression from histological image representations.

• Hu 2026 BRCA2 integrated functional-data classification study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocol. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• Jores plant promoter prediction study

  DNA and genomes

  Original 2021 study comparing GC/motif linear regression and CNN under maize-protoplast and tobacco-leaf reporter conditions.

• LiPP benchmark set

  Protein structure · Molecular interactions

  Benchmark of lipid-protein complex structure prediction: 331 experimental lipid-protein complexes curated from BioDolphin v1.1, with a 36-complex test subset released after every tool's training cutoff.

• M. tuberculosis read classification with standard and custom databases (Hall and Coin 2024)

  Microbes and communities

  Kraken and minimap2 with three databases each, scored per read on simulated and artificial real Nanopore and Illumina sputum-like metagenomes.

• MassSpecGym

  Metabolomics

  MassSpecGym separates spectrum-to-structure generation, candidate retrieval and structure-to-spectrum simulation.

• MIST CANOPUS molecular retrieval

  Metabolomics

  Molecular candidate retrieval from mass spectra, as evaluated in DreaMS Extended Data Table 1. Inputs and extra training data differ across the reported methods.

• Mock community taxonomic profiling benchmark (Portik et al. 2022)

  Microbes and communities

  Eleven profiling methods (14 configurations) on PacBio HiFi, ONT and Illumina reads from ATCC and ZymoBIOMICS mock communities, scored on species detection and abundance.

• MPRabc K562 enhancer–gene evaluation

  Source study/challenge grouping of the exact imported protocols. No claim that this intake implements or reproduces an executable benchmark.

• mRNABench

  RNA and transcriptomes

  mRNABench assesses genomic-model embeddings on transcript-specific expression, stability and regulatory tasks.

• MSAlign molecular retrieval evaluations

  Metabolomics

  Spectrum-to-molecule retrieval comparisons on NPLIB1, Spectraverse and two MassSpecGym splits, reported in MSAlign Table 3. Formula availability and candidate pools remain separate.

• NABench

  RNA and transcriptomes

  NABench compares nucleotide foundation models on measured DNA/RNA sequence effects under multiple adaptation settings.

• Nucleotide Transformer benchmark (18 tasks)

  DNA and genomes

  Eighteen human genomic classification tasks (ten histone marks, enhancers, enhancer types, three promoter sets and three splice-site sets) scored by MCC with tenfold cross-validation.

• OncoVI 2026 oncogenicity evaluation study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• Open Problems

  Single cell

  Open Problems is an extensible platform hosting benchmark tasks and their datasets.

• PerturBench

  Single cell

  PerturBench evaluates predicted single-cell perturbation responses with explicit aggregation and metric choices.

• PEtab benchmark collection

  Mechanistic biology

  The PEtab collection supports evaluation of computational methods for fitting mathematical models to observations.

• PFMBench

  Protein function

  PFMBench is a configurable suite of protein-model downstream evaluations.

• PLABench

  Molecular interactions

  Leakage-controlled, target-centric benchmark of protein-ligand binding affinity prediction: Davis and KIBA under warm, cold-target and cold-drug splits, CASP16 ligand series L1000 and L3000, and a ChEMBL35 test set filtered against the training data of the evaluated models.

• Plant Genomic Benchmark (PGB)

  DNA and genomes

  Plant genomic evaluations introduced with AgroNT. This release contains the complete promoter and terminator comparison tables from Figure 3e and 3f; other tasks remain unextracted.

• Plasma mcfDNA sequencing for predicting and diagnosing bloodstream infection in high-risk leukaemia (Wolf et al. 2026)

  Microbes and communities

  Episode-level sensitivity against blood-culture organisms before and at bloodstream infection onset, and specificity in afebrile control samples.

• PLINDER

  Molecular interactions

  PLINDER supplies annotated protein–ligand systems and evaluation resources for docking.

• PRIME viral mutation-space evaluation

  Protein function

  Evaluation framework for protein representations on SARS-CoV-2 receptor-binding-domain mutants: ACE2 binding and expression regression under a random split and a position-stratified split that holds out mutated positions.

• Protein embeddings for GNN PPI prediction (OGBl-ppa and HuRI)

  Molecular interactions · Proteins and complexes

  Comparison of sequence, structure and function protein embeddings as node features for graph neural network PPI link prediction on OGBl-ppa and HuRI, with a protein-only MLP ablation.

• ProteinBench

  Protein structure

  ProteinBench assesses multiple protein-model tasks using quality, novelty, diversity and robustness dimensions.

• ProteinGym

  Protein function

  ProteinGym separates experimental variant-effect and clinical annotation tasks under supervised and zero-shot regimes.

• RhoFold+ CASP15 natural RNA comparison

  RNA and transcriptomes

  Complete Figure 2h source table: ten configurations, six natural CASP15 RNA targets, RMSD and cumulative GDT-TS/TM Z-score. Retrospective best-of-five comparison; eight unavailable entries retained in extraction receipts.

• RhoMax original wavelength-prediction comparison

  Proteins and complexes

  Original 2024 RhoMax study with four WT-background holdouts and printed aggregate; Bayesian LASSO and geometric neural configurations. Distinct from FLIP2.

• Scaffold-Lab unconditional generation

  Protein structure

  Unified evaluation of protein backbone generation methods: 100 backbones per length group are refolded with ProteinMPNN and ESMFold and scored for designability, novelty, diversity and runtime.

• scFoundation cell-type annotation comparison

  Single cell

  Source-specific reported comparison. Protocol details, fitted configurations and limitations are retained separately from other evaluations.

• scIB

  Single cell

  scIB is an atlas-level single-cell integration benchmark study covering RNA, chromatin-accessibility and simulated datasets. It compares batch removal with preservation of biological variation. The scib Python package and scib-pipeline implement its evaluation workflow.

• SCMBench

  Single cell

  Benchmark of 19 domain-specific models and four single-cell foundation models (scGPT, UCE, Geneformer, scFoundation) for paired and unpaired single-cell multi-omics (RNA and ATAC) integration, scored on integration accuracy, biomarker detection, trajectory conservation and batch correction.

• scTab published annotation evaluation

  Source study/challenge grouping of the exact imported protocols. No claim that this intake implements or reproduces an executable benchmark.

• SegmentNT human genome annotation

  DNA and genomes

  Fourteen genomic-element annotation tasks from the SegmentNT study, with complete per-nucleotide MCC and auPRC comparisons from Supplementary Tables 2 and 3. Each element and metric remains a separate comparison.

• SEQC2 HCC1395 somatic CNV benchmark

  DNA and genomes

  Benchmark of six somatic CNV callers on a hyper-diploid cancer cell line.

• Short-read versus host-depleted nanopore mNGS on BALF (Hu et al. 2026)

  Microbes and communities

  Two clinical metagenomic workflows scored at organism level against culture, clinical microbiological tests and a composite reference standard.

• So 2024 BRCA1 ClinVar conflicting-variant reinterpretation study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocol. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• Static GRN reconstruction from single-cell foundation models (Ye et al. 2026)

  Biological networks · Single cell

  Zero-shot GRN reconstruction from six single-cell foundation models with three extraction strategies (token-embedding cosine, hidden-state cosine, attention) against GENIE3, DeepSEM and GRNBoost, on six BEELINE scRNA-seq datasets and four reference network types.

• Talos 2026 evaluation and reanalysis study

  DNA and genomes

  Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

• TAPE

  Protein function

  TAPE evaluates protein representations through five supervised downstream tasks.

• TDC molecular tasks

  Molecular interactions

  TDC organizes molecular prediction tasks into datasets and benchmark groups with explicit splitting and evaluation interfaces.

• TraitGym

  DNA and genomes

  Causal versus matched non-causal noncoding variants for 113 Mendelian and 83 complex traits, scored by AUPRC computed per chromosome and averaged with sample-size weights.

• Universal antibody-antigen affinity evaluation on AlphaBind (MochiBind study)

  Molecular interactions · Proteins and complexes

  Evaluation framework for antibody-antigen binding affinity on four AlphaBind antigen systems with each antigen held out in turn; pairwise accuracy and retrieval metrics.

• ViroBench

  Microbes and communities · DNA and genomes

  Benchmark of nucleotide foundation models on viral genomics: taxonomy and host classification under genus-disjoint and temporal splits, plus genome modelling and CDS generation.

• Virtual Cell Challenge 2026

  Single cell

  The 2026 Virtual Cell Challenge evaluates perturbation-response prediction in unseen cellular contexts.

• Virus profiler benchmark against a shared Virus Pathogen Database (Tisza et al. 2026)

  Microbes and communities

  EsViritu, Kraken2, Centrifuger, Metabuli and Sylph on positive controls, pathogen-free negative controls and mutated mock communities, all querying the same database.

This index reflects a dated catalogue, not an exhaustive census. Source checking does not mean independent reproduction; compare results only under compatible protocols, datasets and metrics.