Use caseDNA and genomesResearch and clinical research
Review EGFR lung-cancer actionability evidence
Which systems retrieve correctly scoped sensitivity and resistance evidence for advanced EGFR-mutant NSCLC?
What the evidence shows
Two sources add multi-system comparisons on precision-oncology evidence tasks.
Neither is specific to EGFR-mutant NSCLC after targeted therapy, and neither tests treatment choice or patient benefit. Lin et al. 2025 (npj Precision Oncology), whose authors did not develop the models, asked GPT-4o, Llama 3.1 and Qwen 2.5 to assign knowledge-base evidence levels to queries naming a gene, an alteration and a tumour type but no drug. Mean top-1 accuracy over 100 runs in assigning OncoKB levels, which include the resistance levels R1 and R2, was 0.3393, 0.3066 and 0.3328; for CIViC levels it was 0.1865, 0.1212 and 0.2485.
The authors note that the public OncoKB and CIViC tables may be in the models' training data. In the TREC 2020 Precision Medicine track, scored by the track organisers, systems ranked MEDLINE abstracts for 40 cancer, gene and treatment topics, two of them EGFR-mutant NSCLC with afatinib or osimertinib. For topical relevance the best printed runs reached infNDCG 0.5325 (a BM25 baseline tuned on 2019 data) and 0.5303 (BioBERT re-ranking).
For ranking by evidence tier, the best printed automatic run reached NDCG@30 0.4238 (monoT5 with trial and meta-analysis terms added to the query); two manual runs that used human relevance input are not counted. The overview prints only the best run of the top five teams per metric, per-topic EGFR scores are not tabulated, and the topics carry no variant, prior-therapy or resistance context. No source measures retrieval of source-linked resistance evidence after progression on an EGFR inhibitor.
You bring
Interpreted biomarker, genome build/transcript, histology, stage and co-alterations
Prior therapies, progression and sampling dates, including tissue or plasma source
Jurisdiction, evidence cutoff and a versioned source corpus
You want
Source-linked evidence assertions with regimen, sensitivity or resistance direction, treatment setting, native evidence tier, contradictions and unresolved information.
Evidence last checked 9 October 2026 · re-checked monthly, next due 9 November 2026
Executive Takeaway
For "Review EGFR lung-cancer actionability evidence", on LLMs assigning knowledge-base evidence levels (Lin et al. 2025), OncoKB levels 1-4, R1, R2 (Top-1 accuracy, a proxy protocol), GPT-4o scored 0.3393, reported by Benchmarking large language models GPT-4o, llama 3.1, and qwen 2.5 for cancer genetic variant classification.
Best reported valueGPT-4oBenchmarking large language models GPT-4o, llama 3.1, and…
Baseline on this protocolNone scoredNo margin is computed
Ranked tools66 from independent studies
6 tools have a value on this protocol; 6 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.
Ranking
Ranked on LLMs assigning knowledge-base evidence levels (Lin et al. 2025), OncoKB levels 1-4, R1, R2 (Top-1 accuracy, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.
3 tools in 6 rows, from 1 source. 6 of 6 results (100%) come from an independent study.
BM25 keyword retrieval. BM25 keyword ranking is the non-learned retrieval method that the neural rerankers in TREC 2020 Precision Medicine are compared with. The method record is team ASCFDA, whose only run used Anserini BM25 with default parameters; BM25 itself, and the collection plan's knowledgebase lookup with manual review, have no method record.
Qwen2.52 sweepsQwen 2.5 72B, basic prompt, temperature 0.8 (default)foundation model
0.3328
0.2485 (best in column)
Llama 3.13 sweeps70B, basic prompt, temperature 0foundation model
0.3312
Not reported
Caveats and method
What each column measures:
OncoKB levels 1-4, R1, R2: Top-1 accuracy of GPT-4o, Llama 3.1 and Qwen 2.5 (and temperature and prompt variants) in assigning OncoKB levels, including resistance levels R1 and R2, to 625 OncoKB associations
CIViC levels A-E: Top-1 accuracy of GPT-4o, Llama 3.1 and Qwen 2.5 (and a refined Qwen prompt) in assigning CIViC evidence levels to 4,426 CIViC associations
Pan-cancer variant set; no EGFR- or NSCLC-specific score is reported.
The model assigns an evidence level from gene, alteration and tumour type alone; no prior therapy, line or evidence date is given and no source is cited.
Public OncoKB and CIViC tables may be in the models' training data (Discussion P33).
Queries name no drug, so a variant with both a sensitivity and a resistance level for different drugs has no single correct answer; the paper does not say how such queries were scored.
Evidence levels are assigned from the model's own knowledge; except in one FoundationOne run, nothing is retrieved and no source is returned.
Queries name no drug, so a variant with both several evidence items at different levels has no single correct answer; the paper does not say how such queries were scored.
Primary benchmark evaluation shown (LLMs assigning knowledge-base evidence levels (Lin et al. 2025)). 3 additional study protocols (CIViC evidence-direction retrieval, March 2026, CIViC-Fact v3 within-linked-publication passage retrieval, post-cutoff cohort, Precision-oncology literature retrieval systems (TREC 2020 Precision Medicine)) are cited in individual tool drawer rows and under Question, sources and review.
Not covered yet
No independently adjudicated advanced EGFR-mutant NSCLC evidence-retrieval benchmark with prior therapy, treatment line, date, jurisdiction and contradictions was established by this bounded search.
CIViC study reports pan-cancer aggregate and predictive direction metrics; no published EGFR-only score was found.
OncoTraj (arXiv:2606.11144) predicts longitudinal resistance and is outside the requested evidence-retrieval endpoint.
Manual versioned lookup plus primary-source review still needs measured equal effort.
Qualified human scientific review remains outstanding; automated source transcription does not establish experimental replication.
Multi-system comparisons found are pan-cancer: LLM evidence-level assignment (Lin et al. 2025) and literature ranking for cancer, gene and treatment topics (TREC 2020 Precision Medicine, two EGFR topics); neither prints an EGFR-only score.
TREC 2020 Precision Medicine scores for the two EGFR topics appear only as figures in the run appendices, and the topics name a gene and a drug (afatinib, osimertinib) but no variant, prior therapy or resistance question.
LLM evidence-level accuracy may be inflated by training-data exposure to the public OncoKB and CIViC tables (Lin et al. 2025 Discussion), and its queries name no drug, so sensitivity and resistance levels for the same variant are not separated.
TREC 2020 Precision Medicine Phase 2 (evidence-tier ranking) values rest on the overview alone; no public per-run NDCG@30 was found to check them.
No multi-system comparison of retrieving resistance evidence (for example T790M, C797S or MET amplification) after progression on an EGFR inhibitor was found.
Release 2026-11-21-c8bea25e5abb. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.
The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence
Question, sources and review
Who this is for, setting and exclusions
Choose an evidence-review system that connects tumour findings to relevant sensitivity and resistance studies while exposing contradictions and missing context.
Who this is for
Molecular tumour-board scientists
Oncologists and clinical molecular scientists reviewing evidence
Biological setting
Advanced EGFR-mutant non-small-cell lung cancer, including progression after EGFR-targeted therapy. Treatment history and evidence date are part of every comparison.
Clinical research scope
Clinical research on support for qualified molecular-oncology evidence review. Applicability depends on tumour, biomarker, prior therapy, sampling context, jurisdiction and evidence date.
Outside this use case
Evidence retrieval does not select a treatment or establish individual treatment benefit.
Trial eligibility, response and survival require separate evaluations.
Missing evidence does not establish resistance, and evidence tiers from different systems are not interchangeable.
Treatment-recommendation concordance (for example LLM agreement with NCCN guidelines or tumour-board decisions) is treatment selection, not evidence retrieval.
Evidence collection plan
Evidence collection planCollecting evidence
Already on record: 4 comparisons shown · 15 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.
Comparison question
Does retrieval-assisted review recover more correctly scoped sensitivity/resistance evidence with fewer unsupported assertions than a versioned lookup/manual-search workflow at equal review effort?
Baselines, outcomes and validation requirements
Baselines to include
Versioned knowledgebase lookup plus conventional manual primary-source review
Outcomes to measure
Relevant-evidence recall and unsupported-assertion rate
Tumour, treatment-line, direction, date and citation fidelity
Contradiction handling, missing-information handling and review time
Validation requirements
Freeze the question, corpus and knowledge cutoff; hold out studies and molecular profiles.
Preserve native evidence tiers and include contradictory or inapplicable evidence.
Audit citation support and retain unknown clinical information.
Use independent qualified molecular-oncology adjudication.
Confirm source-specific benchmarking permissions and separate evidence review from treatment and eligibility outcomes.
Next collection task
Close the remaining endpoint, independence and comparator gaps documented in the 30 September 2026 audit and linked article issue; obtain qualified scientific review.
Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.