rewirebio.iobenchmarks

The Independent Standard for Biological AIRelease 2026-11-21

Independent Benchmarks for Biological Models

Which biological model works best for your task? Published benchmark results ranked per biological use case, on one protocol at a time, each linked to the source it was taken from.

Per-Use-Case Rankings

No composite scores; models ranked within one protocol per use case.

Protocol Baselines

A baseline appears only where the same study scores it on the same protocol.

Independence and Coverage

Each rank shows who produced the result and how many of the use case's comparisons score the model.

Source Evidence

Every value links to its result record, its source and any open discrepancy.

New. We now flag where studies disagree about the same models, and show what we are doing to resolve each case. See contested results

192 open disagreements between studies, 6 settled in this release.

Use-case Leaderboards

Biological Model Rankings

Each tab ranks models on one protocol of a use case, from the published results in this release. A tab marked Proxy ranks on a protocol that reviewers judged a proxy for the use case, not a direct test of it. Open a row for its source, review status and baseline.

Benchmark Takeaway

For "Predict missense variant effects in a protein of interest", on ProteinGym zero-shot DMS substitutions (commit 144fe22), All assays (Average_Spearman) and breakdowns (Spearman), AIDO Protein-RAG (16B) and VenusREM share the best value, 0.518, reported by ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22. For "Select a tumour DNA somatic variant-calling workflow", on SEQC2 HCC1395 WES validation sample (Guille et al. 2025), SNVs (F1), Lofreq-Muse-Mutect2-SomaticSniper-Strelka-Lancet scored 88.1%, reported by A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing. For "Assess methods for protein stability experiments", on S669 external ddG benchmark (Pancotti et al. 2022), Pearson (S669, 669 direct variants) (Pearson), MAESTRO scored 0.50, reported by Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset. For "Select a tumour RNA fusion-detection workflow", on Validated driver fusions in haematologic cell lines (Tamura et al. 2026), Conventional RNA-seq (Recall), Arriba scored 100%, reported by Comparison of gene fusion detection algorithms reveals frequently overlooked driver fusions in hematologic malignancies. For "Select regulatory variants and genes for functional follow-up", on TraitGym official metrics, Mendelian traits, zero-shot (AUPRC, a proxy protocol), CADD.plus.RawScore scored 0.7122, reported by TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym). For "Use calibrated computational evidence for missense variants (PP3/BP4)", on Livesey and Marsh 2025 VEP rankings, DMS benchmark, all 36 DMS sets (Pairwise win rate, a proxy protocol), CPT-1 scored 92.82%, reported by Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8).

Predict missense variant effects in a protein of interestAIDO Protein-RAG (16B)Spearman 0.518 on ProteinGym zero-shot DMS substitutions (commit…
Select a tumour DNA somatic variant-calling workflowLofreq-Muse-Mutect2-SomaticSniper-Strelka-LancetF1 88.1% on SEQC2 HCC1395 WES validation sample (Guille et…
Assess methods for protein stability experimentsMAESTROPearson 0.50 on S669 external ddG benchmark (Pancotti et al…
Select a tumour RNA fusion-detection workflowArribaRecall 100% on Validated driver fusions in haematologic cell…
Select regulatory variants and genes for functional follow-upCADD.plus.RawScoreAUPRC 0.7122 on TraitGym official metrics
Use calibrated computational evidence for missense variants (PP3/BP4)CPT-1Pairwise win rate 92.82% on Livesey and Marsh 2025 VEP rankings

Each use case is ranked on one protocol only; values are never compared across protocols. Every value links to its result record and source.

Ranked on ProteinGym zero-shot DMS substitutions (commit 144fe22), All assays (Average_Spearman) and breakdowns (Spearman, higher is better). Only results on this protocol are ranked; other tools are listed below the table.

44 tools in 97 rows, from 1 source. 0 of 97 results (0%) come from an independent study; 97 are from a source that does not say who ran the tool. Data release 2026-11-21-adc15377fab2, published 2026-11-21.

Rank▲ModelSpearman (higher is better)vs Site-IndependentEvidenceCoverageReported byLicense
#10.518+44.3%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#10.518+44.3%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#30.507+41.2%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#40.498+38.7%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#50.496+38.2%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#60.488+35.9%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#70.485+35.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#80.479+33.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#90.476+32.6%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#100.471+31.2%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#110.470+30.9%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#110.470+30.9%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#130.469+30.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#140.466+29.8%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…MIT stated in the inspected…
#150.465+29.5%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#160.458+27.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#170.457+27.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#180.456+27.0%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#180.456+27.0%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#200.455+26.7%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#200.455+26.7%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#220.452+25.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#230.449+25.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#240.442+23.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#250.440+22.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#260.439+22.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#260.439+22.3%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#280.438+22.0%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#280.438+22.0%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#280.438+22.0%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#310.437+21.7%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#320.435+21.2%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#330.434+20.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#340.433+20.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#350.432+20.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#350.432+20.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#370.430+19.8%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#380.428+19.2%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#390.424+18.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#400.422+17.5%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Code: MIT
#400.422+17.5%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#420.419+16.7%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#430.418+16.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#440.417+16.2%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#450.414+15.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#460.407+13.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#460.407+13.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Code: MIT
#460.407+13.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#490.406+13.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…MIT is declared alongside…
#490.406+13.1%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#510.405+12.8%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…MIT is declared alongside…
#520.400+11.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#520.400+11.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#540.399+11.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#550.396+10.3%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#560.395+10.0%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#570.394+9.7%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#580.393+9.5%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#590.392+9.2%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#600.391+8.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#610.389+8.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#620.387+7.8%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#630.383+6.7%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#640.380+5.8%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#640.380+5.8%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#660.379+5.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#670.378+5.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#680.374+4.2%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Code: MIT
#680.374+4.2%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#700.373+3.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#700.373+3.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#720.369+2.8%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#730.366+1.9%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#740.364+1.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#740.364+1.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#76
Site-Independent[-] Baseline
0.359Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#770.350-2.5%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#780.349-2.8%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#790.347-3.3%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#790.347-3.3%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#810.343-4.5%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#820.336-6.4%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#830.328-8.6%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#840.321-10.6%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#850.318-11.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#860.317-11.7%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#870.305-15.0%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#880.303-15.6%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#890.282-21.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#900.280-22.0%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#910.257-28.4%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Code: MIT
#920.233-35.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#930.226-37.0%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…The official facebook/esm2_…
#940.190-47.1%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#950.188-47.6%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#960.106-70.5%Source-checked Source does not say who ran it2/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
#970.102-71.6%Source-checked Source does not say who ran it1/2ProteinGym zero-shot DMS substitutions Spearman…Not recorded
71 tools are evaluated elsewhere in this use case but not on this protocol, so not ranked

Search and browse the database

Search in models
More record types
Methods and evaluation records

Biological models and model families. Evaluated versions, configurations and pipelines are linked separately.

Verification and provenance

Audit the evidence

Every score links to its source paper, protocol, and evaluation split (51,847 source-checked published results and 12 results from Rewire runs — source-checked does not mean independently reproduced). Evidence methodology · Collection coverage

Rewire evaluations

Read the protocol, held-out splits and limitations behind our corrected splice-variant evaluation.

MFASS v2 evaluation →
Data access

Download the database

Release 2026-11-21-adc15377fab2 · 2026-11-21

Immutable release exports compressed with gzip and pinned to SHA-256 checksums.