rewirebio.iobenchmarks

Omics and molecular biologyRelease 2026-11-02

Biological model benchmark database

Which biological model works best for your task? Published and reproduced results, linked to their sources and organised by biological question.

New. We now flag where studies disagree about the same models, and show what we are doing to resolve each case. See contested results

121 open disagreements between studies, 5 settled in this release.

Search and browse the database

Search in models
More record types
Methods and evaluation records

Biological models and model families. Evaluated versions, configurations and pipelines are linked separately.

Purpose and scope

What this database is for

01 · The problem

Why comparisons break

Choosing a biological model for a task is harder than it should be. Results are scattered across papers, preprints and benchmark suites. Each uses its own datasets, splits and metrics, so numbers from two papers often cannot be compared, and new models appear faster than anyone can read about them.

02 · Methodology

Organised by biological question

This database brings those results together. Every value links to the paper, table or run it came from. Results are organised around use cases, so you can start from the biological question you are trying to answer rather than from a model name. Results measured under different protocols are kept apart instead of being merged into one ranking, and where the evidence for a use case is thin or missing, the page says so. Where published results leave a gap that matters, we run the benchmark ourselves and publish the code.

03 · Standard

Open, current and correctable

The goal is a trusted, current reference for how biological models perform on each task: kept up to date as the field moves, open to researchers and their AI agents, and correctable by anyone who brings better evidence.

Verification and provenance

How to read the evidence

Published evaluations and rewire evaluations are records in the same database, with their origin shown beside each result. This release includes 49,377 source-checked published results and 12 results from existing rewire runs.

Source checked does not mean independently reproduced. Comparisons require compatible data, protocols and metrics. Missing details stay visible.

Investigate discrepancies

Check whether the data supports replaying a metric, investigating an unexpected result or testing an explanation independently.

Readiness and investigations →

Rewire evaluations

Read the protocol, coverage and limitations behind our corrected splice-variant evaluation.

MFASS v2 evaluation →

Correction history

MFASS v1 is superseded. Its original tables and methods remain available as an archived report.

Archived MFASS v1 report →
Coverage: records and linked evaluations

86,212 records across 17 record types. These counts describe catalogue coverage, not model performance.

Records by kind

  • Results49,393
  • Evaluations22,385
  • Configurations5,315
  • Evidence claims3,340
  • Sources1,377
  • Protocols1,350
  • Datasets732
  • Methods558
  • Dataset subsets553
  • Tasks517
  • Models330
  • Pipelines132
  • Benchmarks105
  • Baselines84
  • Use cases31
  • Services6
  • Evaluators4

Records by research area

  • DNA and genomes14,701
  • RNA and transcriptomes7,247
  • Proteins and complexes6,241
  • Cells and tissues2,468
  • Single-cell omics2,334
  • Microbes and communities2,025
  • Molecular interactions1,808
  • Protein function1,307
  • Biological networks1,288
  • Metabolomics1,143

Benchmark evidence coverage

54 of 105 benchmarks have linked evaluations in this release. Zero means no evaluation is linked here, not that the benchmark has never been used.

  • NABench1,101
  • Virtual Cell Challenge 20261,048
  • mRNABench696
  • ProteinBench556
  • CAFA438
  • BEELINE384
  • Open Problems384
  • GlycanML323
  • DART-Eval316
  • SegmentNT human genome annotation294
  • ProteinGym284
  • GUE280
  • BEACON221
  • FLIP155
  • FLIP2149
  • PFMBench141
  • BEND105
  • CAPRI100
  • HEST-Benchmark100
  • CASP90
  • Gene-MTEB80
  • PerturBench76
  • scIB69
  • TDC molecular tasks66
  • MSAlign molecular retrieval evaluations63
  • ATOM3D57
  • RhoFold+ CASP15 natural RNA comparison56
  • Cancer Immunotherapy Data Science Challenge54
  • TAPE50
  • FoldBench45
  • FramePool translation-prediction study30
  • MassSpecGym24
  • Plant Genomic Benchmark (PGB)24
  • GENEB22
  • scTab published annotation evaluation21
  • Genomic Benchmarks18
  • CAMI16
  • ClusPro Protein Docking Benchmark 5.0 study16
  • RhoMax original wavelength-prediction comparison15
  • scFoundation cell-type annotation comparison14
  • Talos 2026 evaluation and reanalysis study14
  • Genie 3 unconditional short monomer comparison13
  • ESMFold2 Runs N’ Poses comparison11
  • PLINDER7
  • MIST CANOPUS molecular retrieval6
  • MPRabc K562 enhancer–gene evaluation6
  • ENIGMA 2024 BRCA evidence-calibration and curation study5
  • OncoVI 2026 oncogenicity evaluation study5
  • Jores plant promoter prediction study4
  • CIViC MCP 2026 evidence-retrieval evaluation3
  • Benet-Pages 2025 BRCA1/BRCA2 VUS reclassification study2
  • HECTOR 2026 preprint: ENIGMA Evidence Repository validation1
  • Hu 2026 BRCA2 integrated functional-data classification study1
  • So 2024 BRCA1 ClinVar conflicting-variant reinterpretation study1
  • AlphaFold and ESMFold benchmark on PDB structures released 2022 to 20240
  • BERTax taxonomic classification of genome fragments (Mock et al. 2022)0
  • BioML-bench0
  • Brandes et al. 2023 clinical missense pathogenicity benchmarks (ClinVar and HGMD/gnomAD)0
  • Brandes et al. 2023 ClinVar in-frame indel and stop-gain pathogenicity benchmarks0
  • CACHE Challenge #30
  • CASP16 binding affinity challenge0
  • ClinGen SVI calibration of computational predictors for PP3/BP40
  • Davis and KIBA drug-target affinity benchmark (DeepDTA split)0
  • DeepProtein benchmark0
  • DEKOIS 2.00
  • Differential expression classification benchmark for perturbation models (Sun et al. 2026)0
  • DNALONGBENCH0
  • DUD-E (Directory of Useful Decoys, Enhanced)0
  • Evo 2 paper zero-shot human variant effect evaluations (Brixi et al. 2026)0
  • GenBench0
  • GENEB leaderboard0
  • GFMBench-API0
  • GPCRrecent0
  • GPN-MSA variant effect prediction evaluation (Benegas et al.)0
  • GUANinE v1.10
  • Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)0
  • LiPP benchmark set0
  • LIT-PCBA0
  • Livesey and Marsh variant effect predictor benchmark (2025 edition)0
  • Ludaic and Elofsson 2026 RNA structure benchmark0
  • M. tuberculosis read classification with standard and custom databases (Hall and Coin 2024)0
  • MetageNN long-read genus classification benchmark (Peres da Silva et al. 2024)0
  • MF-PCBA0
  • Mock community taxonomic profiling benchmark (Portik et al. 2022)0
  • Nucleotide Transformer benchmark (18 tasks)0
  • PepSpecBench0
  • Perturbation-target prediction from attention-derived edge scores (Kendiukhov 2026, Objective B)0
  • PEtab benchmark collection0
  • PLABench0
  • Plasma mcfDNA sequencing for predicting and diagnosing bloodstream infection in high-risk leukaemia (Wolf et al. 2026)0
  • PRIME viral mutation-space evaluation0
  • Protein embeddings for GNN PPI prediction (OGBl-ppa and HuRI)0
  • Public binding free energy benchmark (Ross et al. 2023)0
  • RNAGym fitness leaderboard v0.1.10
  • Scaffold-Lab unconditional generation0
  • SCMBench0
  • SEQC2 HCC1395 somatic CNV benchmark0
  • Short-read versus host-depleted nanopore mNGS on BALF (Hu et al. 2026)0
  • Static GRN reconstruction from single-cell foundation models (Ye et al. 2026)0
  • TraitGym0
  • Universal antibody-antigen affinity evaluation on AlphaBind (MochiBind study)0
  • ViroBench0
  • Virus profiler benchmark against a shared Virus Pathogen Database (Tisza et al. 2026)0
  • Zero-shot cell embedding evaluation of single-cell foundation models (Kedzierska et al. 2025)0
  • Zero-shot RNA language model benchmark (Wang et al. 2026)0
Collection coverage and remaining gaps

This is a dated collection, not an exhaustive model census. Models, methods, evaluated configurations and pipelines are listed separately. Counts describe records, not unique checkpoints or independent experiments.

entity schema version
1.1
entity migration
baseline release: 2026-09-17-5054ddf2a281; note: Separate models, methods, configurations, pipelines, hosted services, benchmarks, tasks, protocols and evaluators. IDs, printed scores and archived releases remain unchanged.
acquisition review
date: 2026-09-19; scope: Nine benchmark result collections; PEtab timing data remains quarantined. Source cells checked independently; not experimental reproduction.
run instructions
review: Official source instructions; not executed by rewire; guides: 7; documentation audits: 39
run recipe coverage
recipes: 30; evaluations with verified recipe links: 9; top level benchmarks: 105; official documentation or gap: 36; note: Source-reviewed instructions and exact applicability links; execution receipts do not establish reproduction of published scores. Production contributions remain disabled.
research generator sha256
14d01fcbe7cce144376eeb0be5dc527a5a4ec615ba1c039134eebad74ed56beb
discrepancy generator sha256
04187551098f99137222f474a013b4b58f6986d65c13e7f3f17b286169489366
evidence table generator sha256
53e835a4b212bd058c7eef3215a72e4ddfd8f90d47a813f495b28d11c03b39e9
profile coverage
total: 695; fact status counts: source checked: 3883; unreported: 703; unextracted: 49; unavailable: 1; inapplicable: 247; unclassified: 0; note: Source review applies to individual cited claims. Missing fields and inaccessible evidence remain explicit; profile coverage is not independent experimental verification.; reviewed: 92; limited: 603
benchmark paper review
date: 2026-11-02; top level benchmarks: 105; benchmark pages with results: 54; benchmark pages with figures: 38; published comparison figures: 749; source checked result rows: 49377; scope: Counts derived from this release through the production relationship and chart gates. Figures are source-specific, and metric rows are not independent experiments. Source review is not reproduction.
research lanes
9
search entries
645
legacy papers
100
legacy result rows
149
source inputs
file: data/entities/baselines.jsonl; sha256: 0b10107d60b7a7e9d04a120c020e014bb5f6d8824af4f59c0c866cd3cd8333a4; file: data/entities/benchmarks.jsonl; sha256: 3e50b9cfc0a6a6408b85f77a775f6353febc0bf8d01f67dcced53453046e3e1e; file: data/entities/configurations.jsonl; sha256: 060d94e8377e9bdf0c3b8d5d2d2d621c28a8413e2d3abe97b0000bcceb255b6d; file: data/entities/dataset-subsets.jsonl; sha256: d22186aef867e26265a8341d53f9d862c752497800c559e3c76f22f5a4556433; file: data/entities/datasets.jsonl; sha256: d04c7c984b40f210e0f69c140368a78be9cb57384c46409d5d023a434f76781c; file: data/entities/evaluators.jsonl; sha256: ce75e8e4b4c153b0e8d85e861cbc8da2f50ef92325a9829829de44c445d9b2bd; file: data/entities/methods.jsonl; sha256: c382a77225bd4243ffd46e29929510cb36bdb55b214268dac0637e15b57d941d; file: data/entities/models.jsonl; sha256: 1d40931e19de0459e63784868b6ea98bd9cbb64a7986701fa7adc31a3b9bf2f8; file: data/entities/pipelines.jsonl; sha256: 71c672c1e7c257e54c7085937f7d3e3f2ae15d3a2c382f9d736d3b387fff7a86; file: data/entities/protocols.jsonl; sha256: c6d044557e8fc2aed450298ce17770f545b11a084a5ff8832bd3ee38a29b199f; file: data/entities/services.jsonl; sha256: 3de4ff2e14ef5bfffa9bdaacbac1ceb4de01e2f49f168145d48efee33559a1eb; file: data/entities/sources.jsonl; sha256: e6731cb5b9eb5cac914d8940994cbce8418b65a8475070d2060a3c905f778db9; file: data/entities/tasks.jsonl; sha256: 28e8bf5b46494c71d6eed55b48638d8cc165bf734a146b14c3c8bebbd65391f0; file: data/entities/use-cases.jsonl; sha256: 5bca51795e3ba4d4e540e460248bd9fee86e6153c6a5a8c4dbecca1d606eb8b4; file: data/evidence/claims.jsonl; sha256: 470ad2d2d7e98ce5e09796ef8cee5469f00cae3e361c42ad3e3b0105e792e3ac; file: data/evidence/evaluations.jsonl; sha256: d3d4d9f2e50fc3c7d1d2ef072df8907dbc3c685edba8d5e72e29bc2c1156e97c; file: data/evidence/results.jsonl; sha256: 378b0ba0e614c45f76b4dd30a5d0b423050111561327a47106e5c36160ca9654; file: data/provenance/records.jsonl; sha256: 042e3e1ae021cb9d884350ef1b65ce66be82110e68bb09a6938af080a14f76ef; file: data/omics/use-cases/sources/mfass-matched-study-intake.md; sha256: 0af394e74888c01dbe2970d2e28b9bbacb4a83d355881feaaa3a704396a2074d; file: data/omics/use-cases/sources/amfr-pilot-readme.md; sha256: a3208aa4537e8d28c56f68f4559299f31241c50b8d8b2b3776cba5f1718caa43; file: data/omics/use-cases/sources/clinical-priorities-2026-09-28.md; sha256: 57ebf436d7d92d349700d7f3cb533b80b88e4e15d0d3129e824bfa3b06ecdab7; file: data/omics/use-cases/sources/research-priorities-2026-09-28.md; sha256: 1f7c3d2d1e0ba852e4ae725bec2e77e55e0aa81496551242e3fbe9fee7662ae0; file: data/omics/use-cases/review.json; sha256: c97804a941ad497d96d18f24ef944286021cd8603678a1088c45544c834c908e; file: data/research/manifests.json; sha256: bb5b7f5c8c26d1e1318706243135db8f6e9e5afbda48bf87941535f06c4b8662; file: data/research/investigations.json; sha256: 37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570; file: data/omics/search-ledger.jsonl; sha256: 7f4182eb653a603120146dea1492af80d4526ed8cbc3198c4396a9d9568491f2
use cases
schema version: 1.0; input sha256: 43a80aef5473f1a49c013b6d46b720b850ff5a0d18dcbb741dcdeed1ed082db1; use cases: 31; mappings: 462
use case sources
file: use-case-source-0af394e74888c01dbe2970d2e28b9bbacb4a83d355881feaaa3a704396a2074d.md; sha256: 0af394e74888c01dbe2970d2e28b9bbacb4a83d355881feaaa3a704396a2074d; file: use-case-source-1f7c3d2d1e0ba852e4ae725bec2e77e55e0aa81496551242e3fbe9fee7662ae0.md; sha256: 1f7c3d2d1e0ba852e4ae725bec2e77e55e0aa81496551242e3fbe9fee7662ae0; file: use-case-source-57ebf436d7d92d349700d7f3cb533b80b88e4e15d0d3129e824bfa3b06ecdab7.md; sha256: 57ebf436d7d92d349700d7f3cb533b80b88e4e15d0d3129e824bfa3b06ecdab7; file: use-case-source-a3208aa4537e8d28c56f68f4559299f31241c50b8d8b2b3776cba5f1718caa43.md; sha256: a3208aa4537e8d28c56f68f4559299f31241c50b8d8b2b3776cba5f1718caa43
research schema version
1.0
total records
86681
public records
86212
counts
model: 330; method: 558; configuration: 5315; pipeline: 132; service: 6; benchmark: 105; task: 517; protocol: 1350; evaluator: 4; dataset: 732; dataset subset: 553; baseline: 84; evaluation: 22385; result: 49393; source: 1377; claim: 3340; use case: 31
quarantined results
439
scope excluded records
27
source checked results
49377
rewire result rows
12
scope
Specialist omics and molecular models; no clinical assistants or standalone medical imaging.
limitation
A dated discovery and source-transcription review, not an exhaustive census or independent reproduction of external experiments.

Database freshness

Last published update
No verified publication receipt.
Last completed evidence sweep
No completed sweep recorded.
Next planned review
Europe/London · planned · automation not active

Latest attempt: blocked (1 October 2026). This does not advance the completed sweep date.

A monthly sweep does not mean every record was reverified.

Data access

Download the database

Release 2026-11-02-a7f03c9e55a5 · 2026-11-02

Downloads are hosted on GitHub at the pinned data revision and compressed with gzip.

Original literature downloads

The original 100-paper collection is retained for citation history. These files include six result rows excluded from the current omics scope; use the database release above for the reviewed collection.