| Configuration: CADD v1.3 score from the Findlay et al. table | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.804 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.7819119059135827 to 0.8239733581846179 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceCADD v1.3 on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/all/cadd/auroc |
|---|
| Configuration: CADD v1.7 raw score, GRCh37 | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.881 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.8622976465589813 to 0.8970686411150003 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceCADD v1.7 on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/all/auroc/cadd_v1.7/auroc |
|---|
| Configuration: Consequence rule (nonsense and canonical splice, then splice region, then missense, then other) | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.769 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.7483597621938157 to 0.7894260097593949 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceConsequence rule on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/all/consequence_rule/auroc |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Google Cloud H100 | Protocol: Evo 2 7B BRCA1 variant scoring throughput on one GPU (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 2.93 Inference throughput sample-per-second · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B scoring throughput on Google Cloud H100 Workload W1, batch size 1 Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: run receipt, Google Cloud H100 (P2, attempt-1) · evidence/runs/recorded-b75c67aa/platforms/P2/attempt-1/receipt-main.json, JSON pointer /results/w1/sequences_per_second |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod H100 | Protocol: Evo 2 7B BRCA1 variant scoring throughput on one GPU (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 2.91 Inference throughput sample-per-second · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B scoring throughput on Runpod H100 Workload W1, batch size 1 Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: run receipt, Runpod H100 (P3, attempt-1) · evidence/runs/recorded-b75c67aa/platforms/P3/attempt-1/receipt-main.json, JSON pointer /results/w1/sequences_per_second |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod L40S | Protocol: Evo 2 7B BRCA1 variant scoring throughput on one GPU (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.612 Inference throughput sample-per-second · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B scoring throughput on Runpod L40S Workload W1, batch size 1 Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: run receipt, Runpod L40S (P5, attempt-2); evo2-cloud-operations reproduction run 30e7ae49: run receipt, Runpod L40S (P5, attempt-1); evo2-cloud-operations: run log · evidence/runs/recorded-b75c67aa/platforms/P5/attempt-2/receipt-main.json, JSON pointer /results/w1/sequences_per_second |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Google Cloud H100 | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.875 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.8590655608973303 to 0.8908291191943638 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Google Cloud H100) on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/all/evo2_7b/auroc |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod H100 | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.875 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Runpod H100) on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P3/w1_auroc/all |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod L40S | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.877 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Runpod L40S) on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P5/w1_auroc/all |
|---|
| Configuration: Mammalian phyloP from the Findlay et al. table | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.781 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.758980816267467 to 0.8019078820124365 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceMammalian phyloP on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/all/phylop/auroc |
|---|
| Configuration: SpliceAI precomputed masked score from the CADD v1.7 annotations | Protocol: Zero-shot BRCA1 loss-of-function classification, all SNVs (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, all SNVs (evo2-cloud-operations W1) | 0.694 AUROC (LOF against FUNC and INT, all SNVs) unitless · higher Uncertainty: 95% CI 0.6649186773849238 to 0.7233661186966525 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceSpliceAI on BRCA1 all SNVs LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/all/auroc/spliceai/auroc |
|---|