rewirebio.iobenchmarks
← Browse all sources
Source

Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

BioNLP 2025 workshop (ACL) · 2025 · Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30 · Peer-reviewed

DOI: 10.18653/v1/2025.bionlp-1.30

What this source contributed

Records in this release that cite this source. A record can cite several sources.

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Model or configurationMetricValueDatasetEvaluation
GPT-3.5-turbo, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Simple Benchmark)0.42 ± 0.13 fractionVirtual CRISPR Simple BenchmarkGPT-3.5-turbo (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-3.5-turbo, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Simple Benchmark)0.39 ± 0.25 fractionVirtual CRISPR Simple BenchmarkGPT-3.5-turbo (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Simple Benchmark)0.44 ± 0.12 fractionVirtual CRISPR Simple BenchmarkGPT-4 (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Simple Benchmark)0.38 ± 0.21 fractionVirtual CRISPR Simple BenchmarkGPT-4 (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4-turbo, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Simple Benchmark)0.32 ± 0.10 fractionVirtual CRISPR Simple BenchmarkGPT-4-turbo (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4-turbo, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Simple Benchmark)0.15 ± 0.08 fractionVirtual CRISPR Simple BenchmarkGPT-4-turbo (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Simple Benchmark)0.47 ± 0.06 fractionVirtual CRISPR Simple BenchmarkGPT-4o (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Simple Benchmark)0.22 ± 0.06 fractionVirtual CRISPR Simple BenchmarkGPT-4o (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)AUPRC (Difficult Benchmark)N/A unitlessVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)AUROC (Difficult Benchmark)N/A unitlessVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Difficult Benchmark)0.35 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Difficult Benchmark)0.79 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)Negative predictive value (Difficult Benchmark)0.23 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)Precision (Difficult Benchmark)0.32 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)Recall (Difficult Benchmark)0.41 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o, direct prompting, mean over prompting strategies (Song et al. 2025)Specificity (Difficult Benchmark)0.21 fractionVirtual CRISPR Difficult BenchmarkGPT-4o (prompting) on the Virtual CRISPR Difficult Benchmark
GPT-4o-mini, direct prompting, mean over prompting strategies (Song et al. 2025)F1 (Simple Benchmark)0.55 ± 0.04 fractionVirtual CRISPR Simple BenchmarkGPT-4o-mini (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4o-mini, direct prompting, mean over prompting strategies (Song et al. 2025)False positive rate (Simple Benchmark)0.77 ± 0.16 fractionVirtual CRISPR Simple BenchmarkGPT-4o-mini (direct prompting, mean over strategies) on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.48 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.28 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)Negative predictive value (Simple Benchmark)0.65 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)Precision (Simple Benchmark)0.53 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)Recall (Simple Benchmark)0.44 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, without chain-of-thought (Song et al. 2025)Specificity (Simple Benchmark)0.72 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.53 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.25 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)Negative predictive value (Simple Benchmark)0.68 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)Precision (Simple Benchmark)0.58 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)Recall (Simple Benchmark)0.49 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 0-shot, with chain-of-thought (Song et al. 2025)Specificity (Simple Benchmark)0.75 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 0-shot, CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.31 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.09 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)Negative predictive value (Simple Benchmark)0.62 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)Precision (Simple Benchmark)0.62 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)Recall (Simple Benchmark)0.21 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), without chain-of-thought (Song et al. 2025)Specificity (Simple Benchmark)0.91 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.46 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.20 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)Negative predictive value (Simple Benchmark)0.65 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)Precision (Simple Benchmark)0.57 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)Recall (Simple Benchmark)0.39 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (-), with chain-of-thought (Song et al. 2025)Specificity (Simple Benchmark)0.80 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (-), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.49 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.23 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)Negative predictive value (Simple Benchmark)0.66 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)Precision (Simple Benchmark)0.56 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)Recall (Simple Benchmark)0.43 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), without chain-of-thought (Song et al. 2025)Specificity (Simple Benchmark)0.77 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT N, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), with chain-of-thought (Song et al. 2025)F1 (Simple Benchmark)0.50 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT Y, on the Virtual CRISPR Simple Benchmark
GPT-4o, 1-shot (+), with chain-of-thought (Song et al. 2025)False positive rate (Simple Benchmark)0.23 fractionVirtual CRISPR Simple BenchmarkGPT-4o, 1-shot (+), CoT Y, on the Virtual CRISPR Simple Benchmark

Claims

Statements this source supports, each with where in the source it was checked.

  • intended_use: pheno-20261022-method-song2025-virtual-crispr-embedding-classifier: Intended use The authors present in silico prediction of CRISPR screen outcomes as a way to prioritise experiments and guide screen design, and propose using failures of LLM prediction on robust hits to flag novel biology. The evaluation itself is binary classification of pre-selected gene-phenotype queries; in vitro validation of predictions is listed as future work. (Abstract; Section 5; Limitations and Future Work)
  • leakage_control: pheno-20261022-data-song2025-difficult: Leakage control The Difficult Benchmark is built from two publications released in late 2024, selected because they post-date the training cutoffs of all evaluated LLMs, and contains no data from BioGRID-ORCS. (Section 2.1 item (2); Section 3 last sentence; Limitations)
  • leakage_control: pheno-20261022-data-song2025-simple: Leakage control The Simple Benchmark's four screens come from BioGRID-ORCS v1.1.16, the database the embedding classifier is also trained on (the training release is not stated); the authors evaluate the classifier only on the Difficult Benchmark 'to mitigate potential bias from dataset similarity'. Only prompted LLMs are scored on the Simple Benchmark. (Section 2.3 'Simple Benchmark'; Section 3)
  • training_data: pheno-20261022-method-song2025-virtual-crispr-embedding-classifier: Training data Trained on 1,678 BioGRID-ORCS screens in human or mouse cells (from an initial pool of 1,924; about 22.6 million gene-level results; 7.74% hits under the high-confidence definition), with 5 screens held out for validation. The ORCS release used for training is not stated. (Section 3)

Models, benchmarks and other records

Models (13)

Methods (2)

Configurations (30)

Protocols (2)

Datasets (3)

Evaluations (31)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

25 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statementOriginal source and locationReview and provenance
Access and reuse
The ACL Anthology landing page states that material published in or after 2016 is licensed under CC BY 4.0.
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.access_and_reuse

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Archive note
Not archived in the batch; the artifact_sha256 pins the bytes that were read. Text layer kept in the extractor's scratchpad only.
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.archive_note

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Artifact SHA-256
6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.artifact_sha256

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Artifact url
https://aclanthology.org/2025.bionlp-1.30.pdf
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.artifact_url

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Code url
https://github.com/czbiohub-chi/immune-llm-acl
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.code_url

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Doi
10.18653/v1/2025.bionlp-1.30
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.doi

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Evidence concerns
1 values
  • source id
    pheno-20261022-source-song2025
    message
    Appendix A.2 describes the random baseline as predicting hit with probability equal to 'the overall proportion of actual hits in the specific benchmark dataset'. With one pooled rate (485 of 1,175 = 41.3%, Table 3) the expected PPV, sensitivity, F1 and FPR are all about 0.41 and specificity and NPV about 0.59. Table 8 prints PPV 0.47, sensitivity 0.46, F1 0.47, FPR 0.36, specificity 0.64 and NPV 0.63. With each screen's own hit rate (Table 3) the expected values are 0.47, 0.47, 0.47, 0.37, 0.63 and 0.63, which match. One draw at the pooled rate would need sensitivity about 2 standard errors above and FPR about 3 below expectation at once. The baseline therefore probably uses per-screen hit rates, which is information a screen designer would not have before the screen. The expected values were computed by the reviewer from Table 3 counts and are not stored as results. Treat the random row as a per-screen-prevalence classifier, not a matched random-selection comparator.
    source locator
    Appendix A.2; Table 3; Table 1, row "Random Baseline"; Table 8, row "Random N/A N/A"
    artifact sha256
    6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c
    review date
    2026-10-22
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.evidence_concerns

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Extraction method
pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.extraction_method

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Licence
CC-BY-4.0
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.licence

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

Media type
application/pdf
Source metadata		Virtual CRISPR: Can LLMs Predict CRISPR Screen Results?

Original source ↗

No field-specific location recorded

Version: Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
Retrieved: 2026-10-22T13:57:26Z

catalogued

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.media_type

Source artifact SHA-256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c

Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

Primary source retrieved and hashed for the phenotype perturbation selection use-case refresh.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)
Technical metadata and extraction receipts

Stable ID: pheno-20261022-source-song2025

areas
cells-tissues
contexts
research
url
https://doi.org/10.18653/v1/2025.bionlp-1.30
artifact url
https://aclanthology.org/2025.bionlp-1.30.pdf
version
Proceedings of the 24th Workshop on Biomedical Language Processing (BioNLP 2025), pages 354-364, 1 August 2025; ACL Anthology 2025.bionlp-1.30
retrieved at
2026-10-22T13:57:26Z
artifact sha256
6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c
doi
10.18653/v1/2025.bionlp-1.30
publication status
peer_reviewed
licence
CC-BY-4.0
media type
application/pdf
venue
BioNLP 2025 workshop (ACL)
year
2025
access and reuse
The ACL Anthology landing page states that material published in or after 2016 is licensed under CC BY 4.0.
source locator
Full artifact bytes; per-record locators on each record
archive note
Not archived in the batch; the artifact_sha256 pins the bytes that were read. Text layer kept in the extractor's scratchpad only.
extraction method
pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/song2025virtualcrispr.py with row labels asserted
code url
https://github.com/czbiohub-chi/immune-llm-acl
evidence concerns
source id: pheno-20261022-source-song2025; message: Appendix A.2 describes the random baseline as predicting hit with probability equal to 'the overall proportion of actual hits in the specific benchmark dataset'. With one pooled rate (485 of 1,175 = 41.3%, Table 3) the expected PPV, sensitivity, F1 and FPR are all about 0.41 and specificity and NPV about 0.59. Table 8 prints PPV 0.47, sensitivity 0.46, F1 0.47, FPR 0.36, specificity 0.64 and NPV 0.63. With each screen's own hit rate (Table 3) the expected values are 0.47, 0.47, 0.47, 0.37, 0.63 and 0.63, which match. One draw at the pooled rate would need sensitivity about 2 standard errors above and FPR about 3 below expectation at once. The baseline therefore probably uses per-screen hit rates, which is information a screen designer would not have before the screen. The expected values were computed by the reviewer from Table 3 counts and are not stored as results. Treat the random row as a per-screen-prevalence classifier, not a matched random-selection comparator.; source locator: Appendix A.2; Table 3; Table 1, row "Random Baseline"; Table 8, row "Random N/A N/A"; artifact sha256: 6c53ffec88444768fbb5c1a65031c096f07b3d9ac48de8163bc8f47d0d51205c; review date: 2026-10-22
Related records

    Suggest a correction