Herasymenko et al. 2026, J Chem Inf Model (CACHE Challenge #3, Europe PMC full text)

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 1 benchmark
• 1 dataset
• 8 evidence claims
• 1 use case

Claims

Statements this source supports, each with where in the source it was checked.

• CACHE #3 merged-library protocol: normalised columns described by their stored formula: Metric definition and attributes: limitations Every workflow was applied to the merged library of all 1739 Round 1 compounds. Confirmed hits with a score above 15 were clustered into four chemically novel Tier 1 compounds and 25 Tier 2 compounds. Columns L to Q are counts. Columns R, S and T are stored as the formulas 1000*N/L, 1000*O/L and R+S, so they are confirmed Tier 1 or Tier 2 hits per 1000 compounds predicted active; the Figure 6c caption's wording, 'the aggregated score of all compounds predicted active divided by the number of compounds predicted to be actives', does not describe that formula. || ["The library is the union of what the 23 workflows themselves selected, so it is not an independent library and each workflow's own picks are in it.", "Four of the 23 workflows (1694, 1695, 1716, 1717) have no values printed in these columns; they are recorded as absent, not as zero.", "Only 29 confirmed hits scored above 15, four of them chemically novel, so the Tier 1 counts rest on four compounds in total.", "The two normalised columns are labelled as counts of hits while the Figure 6c caption defines the normalised quantity as an aggregated score divided by the number of compounds predicted active. The stored cell formulas (1000*N/L and 1000*O/L) agree with the column labels and not with the caption. The sheet's final column has no printed header at all; its formula is R+S.", "Columns L to Q are counts of compounds, not rates. Columns R, S and T are rates: the workbook stores them as 1000*N/L, 1000*O/L and R+S, so each is confirmed Tier 1 or Tier 2 hits per 1000 compounds predicted active. Predicting the merged library is also not the same task as selecting compounds to synthesise.", "The article is internally inconsistent about Figure 6. The Results section 'Trends and Strategies from the Best Performing Computational Workflows' calls panels a, b and c 'top scoring compounds, top hit rates, and top selections from the merged Round 1 library', while the Figure 6 caption calls panel b 'the aggregated score of all Round 1 hits' and panel c the normalised Tier 1 and Tier 2 counts. The body text therefore calls an aggregated score a hit rate. No value in this batch is taken from Figure 6; every value is from sheet S8."] (Sheet 'S8' columns R, S and T and their stored formulas; article Figure 6 caption and Results 'Trends and Strategies from the Best Performing Computational Workflows')
• CACHE #3 normalised merged-library results: metric_qualifier corrected to the stored formula: Metric qualifier Confirmed Tier 1 (chemically novel) hits among the workflow's predicted actives on the merged 1739-compound library, per 1000 compounds predicted active. The cell is stored as the formula 1000*N/L, that is 1000 times the Tier 1 count in column N divided by the predicted-active count in column L. | Confirmed Tier 2 (not novel) hits among the workflow's predicted actives on the merged 1739-compound library, per 1000 compounds predicted active. The cell is stored as the formula 1000*O/L, that is 1000 times the Tier 2 count in column O divided by the predicted-active count in column L. | Confirmed Tier 1 and Tier 2 hits combined among the workflow's predicted actives on the merged 1739-compound library, per 1000 compounds predicted active. The cell is stored as the formula R+S, the sum of the two preceding columns. (Sheet 'S8', columns R, S and T; the stored cell formulas in xl/worksheets/sheet9.xml)
• CACHE #3 own-selection protocol: hit-rate limitation corrected: Limitations ["Each workflow selected its own compounds, so the workflows are not compared on a common library in this arm. The merged-library arm is the comparison on one library.", "No hit rate is printed for this arm. Compounds tested (column D) and Hits (column E) are printed as separate counts and this batch records both without dividing them, because a derived rate would be a computation rather than a printed value. The article does publish a per-workflow rate, but for the merged-library arm: columns R, S and T of sheet S8 are stored as the formulas 1000*N/L, 1000*O/L and R+S, that is confirmed Tier 1 or Tier 2 hits per 1000 compounds predicted active.", "Workflow numbers of compounds tested range from 41 to 99, so the counts rest on small and unequal denominators.", "The activity and novelty scores are committee judgements, not measurements, and the per-compound scoring matrix (sheet S7) was not extracted.", "Participants are named for only 10 of the 23 workflows in this sheet; the rest are anonymous.", "Twelve of the 23 workflows returned no confirmed hit, and the workbook stores 0 for their score of best hit, aggregated score, top novelty score and top score of novel hits. A 0 in those columns means there was no hit to score, not a hit scored at zero; each affected result carries that warning.", "The article is internally inconsistent about Figure 6. The Results section 'Trends and Strategies from the Best Performing Computational Workflows' calls panels a, b and c 'top scoring compounds, top hit rates, and top selections from the merged Round 1 library', while the Figure 6 caption calls panel b 'the aggregated score of all Round 1 hits' and panel c the normalised Tier 1 and Tier 2 counts. The body text therefore calls an aggregated score a hit rate. No value in this batch is taken from Figure 6; every value is from sheet S8."] (Sheet 'S8' columns D, E, R, S and T and their stored formulas)
• hitdisc-20261022-use-case-mapping-cache3-merged-library: WF1696 wording corrected to the source's own: Rationale Direct, and the closest thing in this batch to the use case's question asked cleanly: every workflow ranked the same 1,739 compounds, all of which had been assayed, so the library is held constant and only the ranking differs. The article's own reading is that WF1696, a purely physics-based workflow, retrieved did better than any other method in retrieving hits from the collection of all molecules selected in Round 1; in table S8 it predicted 99 compounds active and captured 2 of the 4 Tier 1 and 9 of the 25 Tier 2 confirmed hits. (Herasymenko et al. 2026, Results 'Trends and Strategies from the Best Performing Computational Workflows')
• hitdisc-20261022-use-case-summary: WF1696 wording corrected to the source's own: Claimed value Three sources in this store score machine-learning and co-folding methods against docking on the same screening data, and what the numbers say depends on how the negatives were built. On DEKOIS 2.0, whose 79 targets each pair 40 validated actives with 1,200 property-matched decoys selected from ZINC (1:30), the co-folding confidence scores enrich strongly and beat the docking arms. AF3 min-iPAE gives a mean EF1% of 23.15 and Boltz-2 min-iPAE 22.73, against 12.01 for the Glide SP row quoted from another paper, 5.16 for Gnina Vina and 2.94 for TankBind; the best rescoring of a co-folded pose, RTMScore on AF3 poses, reaches 21.34. The authors flag their own Boltz-2 affinity arms (mean EF1% 23.54 and 25.84) as likely over-estimates because Boltz-2 drew affinity data from the same public databases that source DEKOIS 2.0. On GPCRrecent, 16 targets whose first crystal structures were released after 2022 with decoys generated by a conditional recurrent neural network at 1:50, every value falls and the margin narrows: Boltz-2 min-iPAE gives a mean EF1% of 13.96 against 5.02 for Glide SP and 3.65 for Gnina Vina. On the five LIT-PCBA targets where the inactives were experimentally measured, the margin largely disappears: averaged over the five targets Boltz-2 min-iPAE gives EF1% 9.30 and NEF1% 0.100 against Glide SP's 6.79 and 0.079, and on TP53 both Boltz-2 scores give EF1% 0.00 while Protenix ipTM gives 5.00 and Glide SP 3.75. A separate audit of LIT-PCBA reports 323 active training-validation analogue pairs at ECFP4 Tanimoto at or above 0.6 in ALDH1 alone and 2,491 2D-identical inactives shared between training and validation across targets, and concludes that a parameter-free memorisation baseline matches or exceeds published deep-learning results there, so every LIT-PCBA value above carries that limit. Where the negatives are compounds a docking screen already ranked highly and every compound was synthesised and assayed, the ordering changes. On the AmpC list (247 inhibitors among 1,293 tested compounds from a 1.7-billion molecule screen) DOCK3.7 separates binders best, AUC 76.5, against 68.1 for Boltz-2 pIC50 and 60.5 for AF3 L-pLDDT. On the dopamine D4 list (205 binders among 541) docking and Boltz-2 are indistinguishable at the precision printed (both about 71) and AF3 L-pLDDT is worse than random at 46.4. Only on the sigma-2 list (201 binders among 506) does a co-folding score lead, Boltz-2 pIC50 at 83.8 against DOCK3.7's 78.8, with AF3 L-pLDDT at 56.1. In the one blinded prospective challenge, CACHE #3 on the SARS-CoV-2 Nsp3 macrodomain, 23 workflows selected 1,739 compounds that were synthesised and assayed; 28 advanced to hit expansion and 29 confirmed hits scored above 15, four of them judged chemically novel. Twelve of the 23 workflows returned no confirmed hit from their own selection of 41 to 99 compounds. When every workflow was applied to the same merged 1,739-compound library, WF1696, which the article describes as a purely physics-based method, captured 2 of the 4 Tier 1 and 9 of the 25 Tier 2 confirmed hits from 99 predicted actives, which the authors report as did better than any other method in retrieving hits from the collection of all molecules selected in Round 1. Cost bounds all of this: co-folding one protein-ligand pair takes 7.79 to 34.58 seconds per compound on one NVIDIA H100 with pre-computed MSAs, which is why the LIT-PCBA arm covers five targets rather than fifteen. No docking tool was timed on the same hardware, no ligand-similarity or pharmacophore baseline is scored anywhere in this batch, and no result here estimates the hit rate of an untested library. (Herasymenko et al. 2026, Results 'Trends and Strategies from the Best Performing Computational Workflows')
• Relevance of hitdisc-20261022-protocol-cache3-merged-library to use-case-hit-discovery-virtual-screening: Assessed by CACHE #3 merged Round 1 library prediction per workflow (Herasymenko et al. 2026 Table S8) (hitdisc-20261022-protocol-cache3-merged-library: see its source_locator)
• Relevance of hitdisc-20261022-protocol-cache3-own-selection to use-case-hit-discovery-virtual-screening: Assessed by CACHE #3 Round 1 own-selection outcome per workflow (Herasymenko et al. 2026 Table S8) (hitdisc-20261022-protocol-cache3-own-selection: see its source_locator)
• Summary of the evidence for use-case-hit-discovery-virtual-screening: Summary Three sources in this store score machine-learning and co-folding methods against docking on the same screening data, and what the numbers say depends on how the negatives were built. On DEKOIS 2.0, whose 79 targets each pair 40 validated actives with 1,200 property-matched decoys selected from ZINC (1:30), the co-folding confidence scores enrich strongly and beat the docking arms. AF3 min-iPAE gives a mean EF1% of 23.15 and Boltz-2 min-iPAE 22.73, against 12.01 for the Glide SP row quoted from another paper, 5.16 for Gnina Vina and 2.94 for TankBind; the best rescoring of a co-folded pose, RTMScore on AF3 poses, reaches 21.34. The authors flag their own Boltz-2 affinity arms (mean EF1% 23.54 and 25.84) as likely over-estimates because Boltz-2 drew affinity data from the same public databases that source DEKOIS 2.0. On GPCRrecent, 16 targets whose first crystal structures were released after 2022 with decoys generated by a conditional recurrent neural network at 1:50, every value falls and the margin narrows: Boltz-2 min-iPAE gives a mean EF1% of 13.96 against 5.02 for Glide SP and 3.65 for Gnina Vina. On the five LIT-PCBA targets where the inactives were experimentally measured, the margin largely disappears: averaged over the five targets Boltz-2 min-iPAE gives EF1% 9.30 and NEF1% 0.100 against Glide SP's 6.79 and 0.079, and on TP53 both Boltz-2 scores give EF1% 0.00 while Protenix ipTM gives 5.00 and Glide SP 3.75. A separate audit of LIT-PCBA reports 323 active training-validation analogue pairs at ECFP4 Tanimoto at or above 0.6 in ALDH1 alone and 2,491 2D-identical inactives shared between training and validation across targets, and concludes that a parameter-free memorisation baseline matches or exceeds published deep-learning results there, so every LIT-PCBA value above carries that limit. Where the negatives are compounds a docking screen already ranked highly and every compound was synthesised and assayed, the ordering changes. On the AmpC list (247 inhibitors among 1,293 tested compounds from a 1.7-billion molecule screen) DOCK3.7 separates binders best, AUC 76.5, against 68.1 for Boltz-2 pIC50 and 60.5 for AF3 L-pLDDT. On the dopamine D4 list (205 binders among 541) docking and Boltz-2 are indistinguishable at the precision printed (both about 71) and AF3 L-pLDDT is worse than random at 46.4. Only on the sigma-2 list (201 binders among 506) does a co-folding score lead, Boltz-2 pIC50 at 83.8 against DOCK3.7's 78.8, with AF3 L-pLDDT at 56.1. In the one blinded prospective challenge, CACHE #3 on the SARS-CoV-2 Nsp3 macrodomain, 23 workflows selected 1,739 compounds that were synthesised and assayed; 28 advanced to hit expansion and 29 confirmed hits scored above 15, four of them judged chemically novel. Twelve of the 23 workflows returned no confirmed hit from their own selection of 41 to 99 compounds. When every workflow was applied to the same merged 1,739-compound library, WF1696, which the article describes as a purely physics-based method, captured 2 of the 4 Tier 1 and 9 of the 25 Tier 2 confirmed hits from 99 predicted actives, which the authors report as did better than any other method in retrieving hits from the collection of all molecules selected in Round 1. Cost bounds all of this: co-folding one protein-ligand pair takes 7.79 to 34.58 seconds per compound on one NVIDIA H100 with pre-computed MSAs, which is why the LIT-PCBA arm covers five targets rather than fifteen. No docking tool was timed on the same hardware, no ligand-similarity or pharmacophore baseline is scored anywhere in this batch, and no result here estimates the hit rate of an untested library. (Shen et al. 2025 Tables 1-5; Kim, Correy, Hall et al. 2026 Results; Herasymenko et al. 2026 Results and sheet S8; arXiv:2507.21404v2 Abstract)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.