Dataset

Virtual CRISPR BioGRID-ORCS training set

Evidence concern: excluded from comparisons

Appendix A.2 describes the random baseline as predicting hit with probability equal to 'the overall proportion of actual hits in the specific benchmark dataset'. With one pooled rate (485 of 1,175 = 41.3%, Table 3) the expected PPV, sensitivity, F1 and FPR are all about 0.41 and specificity and NPV about 0.59. Table 8 prints PPV 0.47, sensitivity 0.46, F1 0.47, FPR 0.36, specificity 0.64 and NPV 0.63. With each screen's own hit rate (Table 3) the expected values are 0.47, 0.47, 0.47, 0.37, 0.63 and 0.63, which match. One draw at the pooled rate would need sensitivity about 2 standard errors above and FPR about 3 below expectation at once. The baseline therefore probably uses per-screen hit rates, which is information a screen designer would not have before the screen. The expected values were computed by the reviewer from Table 3 counts and are not stored as results. Treat the random row as a per-screen-prevalence classifier, not a matched random-selection comparator.Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? · Appendix A.2; Table 3; Table 1, row "Random Baseline"; Table 8, row "Random N/A N/A"

Training set for the Virtual CRISPR embedding classifier: 1,678 BioGRID-ORCS screens in human or mouse cells.

No reviewed evaluations of this dataset are linked in this release.

Research readiness

0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.

Readiness checks, gaps and artifacts

Release 2026-10-23-5347695500ed · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Metrics are recomputed from the saved predictions: not yet verified

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Metrics are recomputed from the saved predictions: not yet verified
Sample annotations are recorded: not yet verified
Dependence between samples is assessed: not yet verified

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
A pinned run recipe exists: not yet verified
Compute requirements are estimated: not yet verified

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Independent validation data exist: not yet verified
Overlap with training data is checked: not yet verified

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Dataset and evaluation context

A dataset supplies biological observations. The evaluation protocol defines how those observations are split, used and scored.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

6 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statement	Original source and location	Review and provenance
Label semantics High-confidence hit definition; screens reporting both significant positive and negative effects for the same gene-phenotype query were excluded. The full BioGRID-ORCS phenotype note is embedded rather than a parsed phenotype. Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.label_semantics` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact
Population 1,678 screens (from 1,924) performed in human or mouse cells; about 22.6 million gene-level results across 3 perturbation methods, 40,461 genes, 133 cell lines and 613 phenotype descriptions; 7.74% of gene-phenotype pairs are hits Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.population` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact
Source location Section 3 Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.source_locator` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact
Split 5 randomly selected screens held out as a validation set for early stopping and hyperparameter tuning Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.split` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact
Description Training set for the Virtual CRISPR embedding classifier: 1,678 BioGRID-ORCS screens in human or mouse cells. Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `description` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact
Name Virtual CRISPR BioGRID-ORCS training set Context-only references	Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? Original source ↗ Section 3 Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `name` Source artifact SHA-256: `6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

1 source record and release history

Virtual CRISPR: Can LLMs Predict CRISPR Screen Results? · Original source · Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: pheno-20261022-data-song2025-orcs-training

areas: cells-tissues
contexts: research
population: 1,678 screens (from 1,924) performed in human or mouse cells; about 22.6 million gene-level results across 3 perturbation methods, 40,461 genes, 133 cell lines and 613 phenotype descriptions; 7.74% of gene-phenotype pairs are hits
split: 5 randomly selected screens held out as a validation set for early stopping and hyperparameter tuning
label semantics: High-confidence hit definition; screens reporting both significant positive and negative effects for the same gene-phenotype query were excluded. The full BioGRID-ORCS phenotype note is embedded rather than a parsed phenotype.
source locator: Section 3
missing metadata: scope note: reason: unreported; note: The paper does not say whether the four Simple Benchmark screens were removed from the training set; the classifier is evaluated only on the Difficult Benchmark; version: reason: unreported; note: Section 3 says 'From the BioGRID-ORCS database'; only the benchmark source is given as v1.1.16 (Section 2.1)

Related records

Suggest a correction