Relevance of Repeat-assay ceiling to ranking analogues of a hit
- Use case
- Rank close analogues of a hit by binding affinity
- Assessed by
- Repeat-assay reproducibility of relative binding free energies
- Relevance
- Proxy evidence
- Endpoint
- Weighted-average pairwise error and rank correlation between two independent assays of the same compounds
Judgement of how closely this protocol answers the lead-optimisation ranking decision. Drafted by the extracting agent; not reviewed.
1 reviewed evaluation
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group fep-baseline Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Free energy baseline against measured affinities Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Weighted-average pairwise error and rank correlation between two independent assays of the same compounds Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:uc-leadopt-20261022-protocol-ross-experimental-survey Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric kendall-tau Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations4 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records10 values
| The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale This is the noise floor the decision rests on: it says how well relative affinities reproduce between two independent assays of the same compounds, so it bounds what any method could achieve on this kind of data. Its granularity is the same as the FEP+ column's, within-series then weighted over series, and the source states the two columns are directly comparable. It is proxy rather than direct evidence for a different reason: it scores assay reproducibility, not any method's ability to rank analogues, so it is a ceiling to read the methods against and not itself an answer to the decision. Context-only references | The maximal and current accuracy of rigorous protein-ligand binding free energy calculations Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs' Version: Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- The maximal and current accuracy of rigorous protein-ligand binding free energy calculations · Original source · Communications Chemistry 6:222 (2023); Europe PMC full-text XML snapshot PMC10576784
Technical metadata and extraction receipts
Stable ID: uc-leadopt-20261022-judgement-ross-experimental-survey
- field
- links:assessed_by:uc-leadopt-20261022-protocol-ross-experimental-survey
- value
- uc-leadopt-20261022-protocol-ross-experimental-survey
- relevance
- proxy
- endpoint
- Weighted-average pairwise error and rank correlation between two independent assays of the same compounds
- rationale
- This is the noise floor the decision rests on: it says how well relative affinities reproduce between two independent assays of the same compounds, so it bounds what any method could achieve on this kind of data. Its granularity is the same as the FEP+ column's, within-series then weighted over series, and the source states the two columns are directly comparable. It is proxy rather than direct evidence for a different reason: it scores assay reproducibility, not any method's ability to rank analogues, so it is a ceiling to read the methods against and not itself an answer to the decision.
- constraints
- Read these values as a ceiling on achievable accuracy, not as a method's score.; Edgewise errors have no equivalent in this column and are printed as N/A.
- limitations
- Measures assay reproducibility, not prediction.; Pools publicly reported assays with Schrodinger internal project data that is not public.; Only 6 functional-vs-functional comparisons contribute, which the source says makes that category's weighted means less meaningful.; The bootstrap intervals are wider than those of the FEP+ column, so the gap between the two columns is less precisely resolved than the point estimates suggest.
- comparison group
- fep-baseline
- comparison title
- Free energy baseline against measured affinities
- stratum label
- Repeat-assay ceiling
- stratum order
- 2
- headline metric
- kendall-tau
- citation locators
- source id: uc-leadopt-20261022-source-ross2023; locator: Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs'
- source locator
- Tables 1, 2 and 3; Methods 'Accuracy metrics'; Results 'Using FEP to resolve ambiguities in the structural inputs'
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; ai-assisted-source-review; method note: Re-retrieved all three sources and matched each pinned SHA-256. Re-rendered the Boltz-2 supplement with pdftotext -layout (poppler 26.08.0) and matched the text layer's recorded SHA-256. Re-read all 256 cells with a parser written for this review that does not import the extraction script, matching each result to its cell through its own source_locator, and separately read appendix tables 11, 12 and 14 from PNG renderings of PDF pages 29 and 30 at 160 dpi, and Ross Table 3 from the pinned XML markup. Checked printed value, numeric value, locator, metric concept, qualifier, direction, unit, uncertainty and the linked configuration, protocol and dataset, and that every N/A or '-' cell is recorded as missing with a reason.; reviewer: claude; reviewer note: Claude review agent, independent of the extraction agent; no human review claimed; reviewed at: 2026-10-22T13:40:00Z; artifact sha256: f371d2e373306588d11b695b8de21b253a90293d6a15884f635bcf927bdebb1d; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10576784/fullTextXML; note: Independent review 2026-10-22: the quoted wording was matched against the pinned artifact and the locator checked.
- reviewed evaluations
- uc-leadopt-20261022-eval-ross-t3-experimental-survey
- pins
- record id: uc-leadopt-20261022-eval-ross-t3-experimental-survey; fields: status; links; attributes.origin; attributes.comparison; sha256: 0865c70ebdc93081d946dea33819ec9755e0c77d02d35f19a5fd8d8f9dbcf598; record id: uc-leadopt-20261022-protocol-ross-experimental-survey; fields: status; name; description; facets; source_ids; links; attributes; sha256: 1016824f1c5e8a34e060157d5ecd3a25161c8d617b4c62471432f83c654a21cf; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-edgewise-mue; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5f287be84ac1f98a5f93960e4754732deb93528f24c9c4ebda33b3e1a895a575; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-edgewise-rmse; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 38d3168661a6a41e53aace8f47d4235527c1e29ae40382258f4c60493fab1387; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-kendall-tau; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 31f9d21e0042d0c300bf61c43d309c0218a30276ed465a4d730d3282d5fb295d; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-pairwise-mue; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8d836d2d0d5ddc4a85b3e26788b2f45cdbfe11b4f03e9c307147778a20ca43eb; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-pairwise-rmse; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 280f321e87ccd2b851cc6f70cb03a04c1d487ceaedf80af9f78087240dc4e074; record id: uc-leadopt-20261022-result-ross-t3-experimental-survey-r2; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 34df5c4c28250d8011fbc9de7b1177e17cb6a6ea12e7b82d419a786b477e24b0; record id: uc-leadopt-20261022-source-ross2023; fields: status; attributes.artifact_sha256; sha256: a43d6286907e82669a8e328e9f045eb0913afca224a1d7620769936adffd9a2a; record id: use-case-lead-optimisation-affinity-ranking; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 7432deaf6a9ac54d76ccc53abb33ca8b040bf88886f79afd081a08dd686511df
- reason
- First evidence for this use case: the repeat-assay noise floor that bounds what any method could reach, from the primary source, independently reviewed against its pinned artifact on 2026-10-22.