rewirebio.iobenchmarks
Dataset subset

BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)

1125 BRCA1 SNVs from the Findlay et al. 2018 table as used by workload W1, of which 239 are labelled loss of function.

Evaluation results

8 evaluations · 8 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: CADD v1.3 score from the Findlay et al. tableProtocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.875 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.8389465183926725 to 0.9091480784331197

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

CADD v1.3 on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/noncoding/cadd/auroc
Configuration: CADD v1.7 raw score, GRCh37Protocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.932 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.9057216538865969 to 0.9550250561794653

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

CADD v1.7 on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/noncoding/auroc/cadd_v1.7/auroc
Configuration: Consequence rule (nonsense and canonical splice, then splice region, then missense, then other)Protocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.879 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.8492821105698829 to 0.9068297056751355

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

Consequence rule on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/noncoding/consequence_rule/auroc
Configuration: Evo 2 7B zero-shot likelihood score on Google Cloud H100Protocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.953 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.9354191281601245 to 0.967935275020978

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

Evo 2 7B (Google Cloud H100) on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/noncoding/evo2_7b/auroc
Configuration: Evo 2 7B zero-shot likelihood score on Runpod H100Protocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.953 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: Not reported by the source: The file gives no interval at this pointer.

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

Evo 2 7B (Runpod H100) on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P3/w1_auroc/noncoding
Configuration: Evo 2 7B zero-shot likelihood score on Runpod L40SProtocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.953 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: Not reported by the source: The file gives no interval at this pointer.

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

Evo 2 7B (Runpod L40S) on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P5/w1_auroc/noncoding
Configuration: Mammalian phyloP from the Findlay et al. tableProtocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.887 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.8530053374933787 to 0.9186797101555415

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

Mammalian phyloP on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/noncoding/phylop/auroc
Configuration: SpliceAI precomputed masked score from the CADD v1.7 annotationsProtocol: Zero-shot BRCA1 loss-of-function classification, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
Dataset subset: BRCA1 saturation genome editing SNVs, noncoding SNVs (all other consequences) (evo2-cloud-operations W1)
0.963 AUROC (LOF against FUNC and INT, noncoding SNVs (all other consequences))
unitless · higher

Uncertainty: 95% CI 0.9445438804974307 to 0.9795283161421778

Coverage: Not reported scored / Not reported eligible

Rewire evaluation · Source checked
Methods, coverage and source

SpliceAI on BRCA1 noncoding SNVs (all other consequences)

LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function

Aggregation: Not reported

evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/noncoding/auroc/spliceai/auroc

Source checking is not independent reproduction. Release 2026-11-21-adc15377fab2.

Research readiness

0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.

Readiness checks, gaps and artifacts

Release 2026-11-21-adc15377fab2 · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Metrics are recomputed from the saved predictions: not yet verified

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Metrics are recomputed from the saved predictions: not yet verified
  • Sample annotations are recorded: not yet verified
  • Dependence between samples is assessed: not yet verified

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • A pinned run recipe exists: not yet verified
  • Compute requirements are estimated: not yet verified

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Independent validation data exist: not yet verified
  • Overlap with training data is checked: not yet verified

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Subset and evaluation context

This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

14 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance
Eligible count
1125
Context-only references
evo2-cloud-operations amendment 7: current predictors results.json

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.eligible_count

Source artifact SHA-256: 403437941f5a0e08b3730fa7bbfcba0b8e616b0e87b2f8e3f227b2f86a342318

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Eligible count
1125
Context-only references
evo2-cloud-operations: protocol.md

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.eligible_count

Source artifact SHA-256: e24ab6967b409dcf8663b5b9d8917b85ba91146987089240206dfa737b25e2f7

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Positive count
239
Context-only references
evo2-cloud-operations amendment 7: current predictors results.json

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.positive_count

Source artifact SHA-256: 403437941f5a0e08b3730fa7bbfcba0b8e616b0e87b2f8e3f227b2f86a342318

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Positive count
239
Context-only references
evo2-cloud-operations: protocol.md

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.positive_count

Source artifact SHA-256: e24ab6967b409dcf8663b5b9d8917b85ba91146987089240206dfa737b25e2f7

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Source dataset label
Findlay et al. 2018 BRCA1 saturation genome editing table, as copied in ArcInstitute/evo2
Context-only references
evo2-cloud-operations amendment 7: current predictors results.json

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_dataset_label

Source artifact SHA-256: 403437941f5a0e08b3730fa7bbfcba0b8e616b0e87b2f8e3f227b2f86a342318

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Source dataset label
Findlay et al. 2018 BRCA1 saturation genome editing table, as copied in ArcInstitute/evo2
Context-only references
evo2-cloud-operations: protocol.md

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_dataset_label

Source artifact SHA-256: e24ab6967b409dcf8663b5b9d8917b85ba91146987089240206dfa737b25e2f7

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Source location
evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof
Context-only references
evo2-cloud-operations amendment 7: current predictors results.json

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_locator

Source artifact SHA-256: 403437941f5a0e08b3730fa7bbfcba0b8e616b0e87b2f8e3f227b2f86a342318

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Source location
evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof
Context-only references
evo2-cloud-operations: protocol.md

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_locator

Source artifact SHA-256: e24ab6967b409dcf8663b5b9d8917b85ba91146987089240206dfa737b25e2f7

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Version
f8ae11c9a1c45505487c978b4cb7e064c4e59905
Context-only references
evo2-cloud-operations amendment 7: current predictors results.json

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.version

Source artifact SHA-256: 403437941f5a0e08b3730fa7bbfcba0b8e616b0e87b2f8e3f227b2f86a342318

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

Version
f8ae11c9a1c45505487c978b4cb7e064c4e59905
Context-only references
evo2-cloud-operations: protocol.md

Original source ↗

evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: f8ae11c9a1c45505487c978b4cb7e064c4e59905
Retrieved: 2026-11-21T14:00:00Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.version

Source artifact SHA-256: e24ab6967b409dcf8663b5b9d8917b85ba91146987089240206dfa737b25e2f7

Hash scope: complete bytes of the file at the pinned commit

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: source checked

2 source records and release historyDownload this release (gzip)
Technical metadata and record identifiers

Technical metadata for this record is tracked under stable ID rewire-local-20261121-evo2ops-dataset-brca1-noncoding.

Source locator: evidence/current-predictors/results.json, JSON pointers /main/noncoding/n and /main/noncoding/lof

Related records

Suggest a correction