| Configuration: CADD v1.3 score from the Findlay et al. table | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.841 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.8210657194316037 to 0.8599321977094521 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceCADD v1.3 on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/coding/cadd/auroc |
|---|
| Configuration: CADD v1.7 raw score, GRCh37 | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.892 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.8745825151206439 to 0.907841119980342 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceCADD v1.7 on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/coding/auroc/cadd_v1.7/auroc |
|---|
| Configuration: Consequence rule (nonsense and canonical splice, then splice region, then missense, then other) | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.704 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.6867445703911983 to 0.720968351649087 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceConsequence rule on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/coding/consequence_rule/auroc |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Google Cloud H100 | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.84 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.8180866084646268 to 0.8608278896043638 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Google Cloud H100) on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/coding/evo2_7b/auroc |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod H100 | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.84 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Runpod H100) on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P3/w1_auroc/coding |
|---|
| Configuration: Evo 2 7B zero-shot likelihood score on Runpod L40S | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.843 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: Not reported by the source: The file gives no interval at this pointer. Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceEvo 2 7B (Runpod L40S) on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /platforms/P5/w1_auroc/coding |
|---|
| Configuration: Mammalian phyloP from the Findlay et al. table | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.736 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.7093475524660791 to 0.7624285129432113 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceMammalian phyloP on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations recorded run b75c67aa: results.json · evidence/runs/recorded-b75c67aa/results.json, JSON pointer /brca1_evo2_vs_baselines/coding/phylop/auroc |
|---|
| Configuration: SpliceAI precomputed masked score from the CADD v1.7 annotations | Protocol: Zero-shot BRCA1 loss-of-function classification, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) Dataset subset: BRCA1 saturation genome editing SNVs, coding SNVs (missense, nonsense, synonymous) (evo2-cloud-operations W1) | 0.588 AUROC (LOF against FUNC and INT, coding SNVs (missense, nonsense, synonymous)) unitless · higher Uncertainty: 95% CI 0.5588775138834077 to 0.618192301934167 Coverage: Not reported scored / Not reported eligible | Rewire evaluation · Source checkedMethods, coverage and sourceSpliceAI on BRCA1 coding SNVs (missense, nonsense, synonymous) LOF against FUNC and INT; AUROC of the score oriented so that higher means loss of function Aggregation: Not reported evo2-cloud-operations amendment 7: current predictors results.json · evidence/current-predictors/results.json, JSON pointer /main/coding/auroc/spliceai/auroc |
|---|