rewirebio.iobenchmarks
Evidence claim

Relevance of CIViC-Fact v3 within-linked-publication passage retrieval, post-cutoff cohort to "Review EGFR lung-cancer actionability evidence"

Use case
Review EGFR lung-cancer actionability evidence
Assessed by
CIViC-Fact v3 within-linked-publication passage retrieval, post-cutoff cohort
Relevance
Proxy evidence
Endpoint
CIViC-Fact v3 post-cutoff within-linked-publication passage-retrieval appropriate-content rate (manual review), two retriever configurations on the identical 40-entry cohort

A pan-cancer, within-known-publication passage-retrieval evaluation; not a validated EGFR/NSCLC evidence-retrieval system, and not a source-search or citation-context benchmark across an open literature corpus.

2 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

15 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    ucc-clinical-egfr-source-civicfact-v3
    locator
    Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
3 values
  • 150 candidate CIViC entries submitted/revised 2026-03-03 to 2026-06-09; single temporal cohort, not independent replication.
  • 40/150 scored after sequential exclusions (68 full-text inaccessible, 14 already present in train/dev/test data, 2 data errors, 24 NEI requiring supplementary/image evidence, 2 NEI substantially revised claims); partial, enriched-cohort coverage.
  • Manual review judgment of 'appropriate' retrieved content, not an automated or recomputed metric; reviewer count, blinding and replicate/seed structure for this judgment are unreported, and uncertainty is unreported.
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
CIViC-Fact v3 post-cutoff within-linked-publication passage-retrieval appropriate-content rate (manual review), two retriever configurations on the identical 40-entry cohort
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:ucc-clinical-egfr-protocol-civicfact-v3-retrieval
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
5 values
  • No EGFR-specific subgroup or score anywhere in this cohort.
  • Enriched cohort per the source's own Discussion; not representative of CIViC overall.
  • Retrieval ranks passages within each entry's already-linked source publication, not open-corpus literature search.
  • Does not close the benchmark-execution gap tracked separately at rewire-benchmarks #28.
  • Not the static v2 SUPPORTS/REFUTES/NEI stance-classification accuracy (89%/85%) or the v2-era BGE retriever manual-review figure (70/75); those are distinct, version-specific measurements and are not represented by this mapping.
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
7 values
  • record id
    ucc-clinical-egfr-eval-medcpt-ft-postcutoff
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    874989ebd7e3a045efed840a9e7d9930e9ae018caadc6466c86c4822d61802ba
  • record id
    ucc-clinical-egfr-eval-qwen3-8b-postcutoff
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    6a5c300340e3340d6951b7266146542f036792e6a07e8d872090d5d653fcbf7a
  • record id
    ucc-clinical-egfr-protocol-civicfact-v3-retrieval
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    9725dfcf2b0ecca60de3c4cdd81bd78e253ef0158d3db195993b644ee4e13274
  • record id
    ucc-clinical-egfr-result-medcpt-ft-postcutoff-appropriate
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d828f034804faf0e12f8674822ae51b233b750c57a6c8e8e33e9b0492258c3ae
  • record id
    ucc-clinical-egfr-result-qwen3-8b-postcutoff-appropriate
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d828f034804faf0e12f8674822ae51b233b750c57a6c8e8e33e9b0492258c3ae
  • record id
    ucc-clinical-egfr-source-civicfact-v3
    fields
    • status
    • attributes.artifact_sha256
    sha256
    cd9e37090b217132e403f4a9732ff8e0cf3a174a96c2440b51790464dc3e0a8a
  • record id
    use-case-egfr-nsclc-actionability-resistance-evidence
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    804054066dce49f26b29eaf88d3019a268dcb5771f67fd19889703d1fc157100
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
A pan-cancer, within-known-publication passage-retrieval evaluation; not a validated EGFR/NSCLC evidence-retrieval system, and not a source-search or citation-context benchmark across an open literature corpus.
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Reason
Add a second pan-cancer proxy: CIViC-Fact v3's post-cutoff within-linked-publication passage-retrieval evaluation. Independently reviewed by Codex across four review cycles before ingestion; retains the existing CIViC MCP proxy mapping unchanged.
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.reason

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relevance
proxy
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.relevance

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Revision
1
Context-only references
CIViC-Fact: a proof-of-concept framework for AI-assisted verification of cancer variant interpretations (bioRxiv v3)

Original source ↗

ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)

Version: bioRxiv preprint doi:10.1101/2025.09.10.675443, Version 3, posted 2026-08-29 (per in-document header and bioRxiv API version metadata); published: NA.
Retrieved: 2026-10-07T09:28:49Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.revision

Source artifact SHA-256: 6beb79ede82f7263a06eafc844bcb4ae41538323e3c9a9a7fa5420f08a8add55

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-20261007-345-7c4af3e091bd

field
links:assessed_by:ucc-clinical-egfr-protocol-civicfact-v3-retrieval
value
ucc-clinical-egfr-protocol-civicfact-v3-retrieval
relevance
proxy
endpoint
CIViC-Fact v3 post-cutoff within-linked-publication passage-retrieval appropriate-content rate (manual review), two retriever configurations on the identical 40-entry cohort
rationale
A pan-cancer, within-known-publication passage-retrieval evaluation; not a validated EGFR/NSCLC evidence-retrieval system, and not a source-search or citation-context benchmark across an open literature corpus.
constraints
150 candidate CIViC entries submitted/revised 2026-03-03 to 2026-06-09; single temporal cohort, not independent replication.; 40/150 scored after sequential exclusions (68 full-text inaccessible, 14 already present in train/dev/test data, 2 data errors, 24 NEI requiring supplementary/image evidence, 2 NEI substantially revised claims); partial, enriched-cohort coverage.; Manual review judgment of 'appropriate' retrieved content, not an automated or recomputed metric; reviewer count, blinding and replicate/seed structure for this judgment are unreported, and uncertainty is unreported.
limitations
No EGFR-specific subgroup or score anywhere in this cohort.; Enriched cohort per the source's own Discussion; not representative of CIViC overall.; Retrieval ranks passages within each entry's already-linked source publication, not open-corpus literature search.; Does not close the benchmark-execution gap tracked separately at rewire-benchmarks #28.; Not the static v2 SUPPORTS/REFUTES/NEI stance-classification accuracy (89%/85%) or the v2-era BGE retriever manual-review figure (70/75); those are distinct, version-specific measurements and are not represented by this mapping.
revision
1
reason
Add a second pan-cancer proxy: CIViC-Fact v3's post-cutoff within-linked-publication passage-retrieval evaluation. Independently reviewed by Codex across four review cycles before ingestion; retains the existing CIViC MCP proxy mapping unchanged.
citation locators
source id: ucc-clinical-egfr-source-civicfact-v3; locator: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)
source locator
ucc-clinical-egfr-source-civicfact-v3: Results, 'Passage Retrieval Models Perform Well without Fine-Tuning' subsection (printed p.14 / PDF p.15); selection chain in Methods, 'Temporal evaluation' subsection and Supplementary Figure 2 (printed p.33 / PDF p.34)
review
method: automated-source-review; reviewer: codex; claude-sonnet; reviewer note: Claude Sonnet EGFR-evidence-research worker, independently reviewed by Codex across four review cycles; reviewed at: 2026-10-07T10:26:56.000Z; note: Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.
reviewed evaluations
ucc-clinical-egfr-eval-medcpt-ft-postcutoff; ucc-clinical-egfr-eval-qwen3-8b-postcutoff
pins
record id: ucc-clinical-egfr-eval-medcpt-ft-postcutoff; fields: status; links; attributes.origin; attributes.comparison; sha256: 874989ebd7e3a045efed840a9e7d9930e9ae018caadc6466c86c4822d61802ba; record id: ucc-clinical-egfr-eval-qwen3-8b-postcutoff; fields: status; links; attributes.origin; attributes.comparison; sha256: 6a5c300340e3340d6951b7266146542f036792e6a07e8d872090d5d653fcbf7a; record id: ucc-clinical-egfr-protocol-civicfact-v3-retrieval; fields: status; name; description; facets; source_ids; links; attributes; sha256: 9725dfcf2b0ecca60de3c4cdd81bd78e253ef0158d3db195993b644ee4e13274; record id: ucc-clinical-egfr-result-medcpt-ft-postcutoff-appropriate; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d828f034804faf0e12f8674822ae51b233b750c57a6c8e8e33e9b0492258c3ae; record id: ucc-clinical-egfr-result-qwen3-8b-postcutoff-appropriate; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d828f034804faf0e12f8674822ae51b233b750c57a6c8e8e33e9b0492258c3ae; record id: ucc-clinical-egfr-source-civicfact-v3; fields: status; attributes.artifact_sha256; sha256: cd9e37090b217132e403f4a9732ff8e0cf3a174a96c2440b51790464dc3e0a8a; record id: use-case-egfr-nsclc-actionability-resistance-evidence; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 804054066dce49f26b29eaf88d3019a268dcb5771f67fd19889703d1fc157100
Related records

Suggest a correction