Relevance of Public FEP benchmark to ranking analogues of a hit
- Use case
- Rank close analogues of a hit by binding affinity
- Assessed by
- Public FEP benchmark, FEP+ accuracy against experiment
- Relevance
- Direct evidence
- Endpoint
- Weighted-average pairwise error and rank correlation of a free energy perturbation workflow against measured relative affinities over more than 50 proteins' congeneric series
Judgement of how closely this protocol answers the lead-optimisation ranking decision. Drafted by the extracting agent; not reviewed.
1 reviewed evaluation
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group fep-baseline Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Free energy baseline against measured affinities Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints4 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Weighted-average pairwise error and rank correlation of a free energy perturbation workflow against measured relative affinities over more than 50 proteins' congeneric series Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:uc-leadopt-20261022-protocol-ross-fep-plus Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric kendall-tau Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations5 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records10 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale This is the free energy baseline the use case has to beat, and it is measured on the use case's own endpoint: each error and rank correlation is computed inside one congeneric series, then averaged over series with the weight equal to the number of compounds in the series. It is not a correlation pooled over unrelated targets, so it does not carry the inflation pooling produces. The granularity is series-average: the values support 'on average over these congeneric series, this method ranked analogues this well', and do not support any statement about a specific series or about variance across series, because no per-series value and no across-series spread is printed. Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- The maximal and current accuracy of rigorous protein-ligand binding free energy calculations · Original source · Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784
Technical metadata and extraction receipts
Stable ID: uc-leadopt-20261022-judgement-ross-fep-plus
- field
- links:assessed_by:uc-leadopt-20261022-protocol-ross-fep-plus
- value
- uc-leadopt-20261022-protocol-ross-fep-plus
- relevance
- direct
- endpoint
- Weighted-average pairwise error and rank correlation of a free energy perturbation workflow against measured relative affinities over more than 50 proteins' congeneric series
- rationale
- This is the free energy baseline the use case has to beat, and it is measured on the use case's own endpoint: each error and rank correlation is computed inside one congeneric series, then averaged over series with the weight equal to the number of compounds in the series. It is not a correlation pooled over unrelated targets, so it does not carry the inflation pooling produces. The granularity is series-average: the values support 'on average over these congeneric series, this method ranked analogues this well', and do not support any statement about a specific series or about variance across series, because no per-series value and no across-series spread is printed.
- constraints
- Do not compare these cells with the FEP+ cells quoted in the AEV-PLIG paper (Pearson 0.68, Kendall tau 0.49): those are a different metric and probably a different subset.; Do not compare Ross's weighted averages with the Boltz-2 supplement's assay-weighted target averages as if they were the same aggregation.; Keep the pairwise and edgewise error families separate; the source states the edgewise errors depend on graph topology.; Do not combine these series-average values with per-series FEP+ values recovered from the public benchmark repository: that would mix two granularities in one comparison.
- limitations
- The FEP+ protocol was tuned per system after seeing the error against experiment, so this is a maximum-attainable rather than a prospective baseline.; Reported by the developers of the workflow being assessed.; No per-series values and no across-series spread are printed, so the resolution needed to judge one series is absent.; No compute cost is reported, so the accuracy-against-cost half of the decision cannot be answered from this protocol.; Running the workflow needs a commercial licence, so its values can only ever be recorded from publication.
- comparison group
- fep-baseline
- comparison title
- Free energy baseline against measured affinities
- stratum label
- Public FEP benchmark
- stratum order
- 1
- headline metric
- kendall-tau
- citation locators
- source id: uc-leadopt-20261022-source-ross2023; locator: Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs'
- source locator
- Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs'
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; ai-assisted-source-review; method note: Re-retrieved all three sources and matched each pinned SHA-256. Re-rendered the Boltz-2 supplement with pdftotext -layout (poppler 26.08.0) and matched the text layer's recorded SHA-256. Re-read all 256 cells with a parser written for this review that does not import the extraction script, matching each result to its cell through its own source_locator, and separately read appendix tables 11, 12 and 14 from PNG renderings of PDF pages 29 and 30 at 160 dpi, and Ross Table 3 from the pinned XML markup. Checked printed value, numeric value, locator, metric concept, qualifier, direction, unit, uncertainty and the linked configuration, protocol and dataset, and that every N/A or '-' cell is recorded as missing with a reason.; reviewer: claude; reviewer note: Claude review agent, independent of the extraction agent; no human review claimed; reviewed at: 2026-10-22T13:40:00Z; artifact sha256: f371d2e373306588d11b695b8de21b253a90293d6a15884f635bcf927bdebb1d; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10576784/fullTextXML; note: Independent review 2026-10-22: the quoted wording was matched against the pinned artifact and the locator checked.
- reviewed evaluations
- uc-leadopt-20261022-eval-ross-t3-fep-plus
- pins
- record id: uc-leadopt-20261022-eval-ross-t3-fep-plus; fields: status; links; attributes.origin; attributes.comparison; sha256: bb1d4cdff37a8211eef31fdfae09658f692638365e974478a5e73e77dfb7696d; record id: uc-leadopt-20261022-protocol-ross-fep-plus; fields: status; name; description; facets; source_ids; links; attributes; sha256: 6024dc74fa0f7c2814f67ad3c613801018b3ff6dddeec4d57547da18b9dab2dd; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-edgewise-mue; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d0d1819873217dd94df9635d115d4360a537b0a8965eab7986600033b131532f; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-edgewise-rmse; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2d1d875fe487a68ba0f7569eb5c45a0ede12054a58b22b27d1431c6df02596b5; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-kendall-tau; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cc701093f21a2a56265bf7dbb22eb2f6af607a8f3cb288823cc08210b5370296; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-pairwise-mue; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 72467c89adf69f0a3144205086369d26a0fdee9405a1c946fa5e0c6063d37e4b; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-pairwise-rmse; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5e8cadd183b1fdfdd5205365d12884d518afee1f8254df7d0f2264bdc733d909; record id: uc-leadopt-20261022-result-ross-t3-fep-plus-r2; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: bbaa5e91e028253225f369dc4951a1b12b729f37233d076c6d2d6be73e3fdac0; record id: uc-leadopt-20261022-source-ross2023; fields: status; attributes.artifact_sha256; sha256: a43d6286907e82669a8e328e9f045eb0913afca224a1d7620769936adffd9a2a; record id: use-case-lead-optimisation-affinity-ranking; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 7432deaf6a9ac54d76ccc53abb33ca8b040bf88886f79afd081a08dd686511df
- reason
- First evidence for this use case: the free energy baseline the decision is read against, from the primary source, independently reviewed against its pinned artifact on 2026-10-22.