GRNBoost2 and PPCOR (partial Pearson correlation): BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 reverses the order
- Medium severity
- Not yet reviewed
- Ordering reversal
- Same task, different protocols
BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 ranks PPCOR (partial Pearson correlation) above GRNBoost2 in 9 of 12 comparisons; 2 of 2 other studies rank GRNBoost2 above PPCOR (partial Pearson correlation).
An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.
Which studies agree
2 papers rank GRNBoost2 above PPCOR (partial Pearson correlation) in 47 of their 59 comparisons and PPCOR (partial Pearson correlation) higher in 8; 4 are ties. BEELINE ranks PPCOR (partial Pearson correlation) above GRNBoost2 in 9 of its 12 comparisons and GRNBoost2 higher in 3.
- Rank GRNBoost2 higher (2): BEELINE, BEELINE
- Rank PPCOR (partial Pearson correlation) higher (1): BEELINE (disagrees)
- Mixed (0): none
Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.
| Paper | Role | Ranks GRNBoost2 higher | Ranks PPCOR (partial Pearson correlation) higher | Ties |
|---|---|---|---|---|
| BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 · Source | Disagrees | 3 | 9 | 0 |
| BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 · Source | Agrees | 37 | 6 | 1 |
| BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 · Source | Agrees | 10 | 2 | 3 |
Comparisons in BEELINE (12)
| Protocol | Dataset | Metric | GRNBoost2 | PPCOR (partial Pearson correlation) | Higher |
|---|---|---|---|---|---|
| BEELINE 2020 Figure 2 · LI | LI | AUPRC ratio, unitless | 2.09 | 2.8 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · LI | LI | Beeline stability, fraction | 0.556 | 0.6 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · TF | TF | AUPRC ratio, unitless | 1.62 | 1.75 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · TF | TF | Beeline stability, fraction | 0.7 | 0.718 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · CY | CY | AUPRC ratio, unitless | 1.68 | 1.56 | GRNBoost2 |
| BEELINE 2020 Figure 2 · CY | CY | Beeline stability, fraction | 0.714 | 0.5 | GRNBoost2 |
| BEELINE 2020 Figure 2 · BFC | BFC | AUPRC ratio, unitless | 2.24 | 2.57 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · BFC | BFC | Beeline stability, fraction | 0.667 | 0.778 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · LL | LL | AUPRC ratio, unitless | 6.16 | 7.72 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · LL | LL | Beeline stability, fraction | 0.44 | 0.636 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · BF | BF | AUPRC ratio, unitless | 1.39 | 1.77 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 2 · BF | BF | Beeline stability, fraction | 0.603 | 0.429 | GRNBoost2 |
Comparisons in BEELINE (44)
Comparisons in BEELINE (15)
| Protocol | Dataset | Metric | GRNBoost2 | PPCOR (partial Pearson correlation) | Higher |
|---|---|---|---|---|---|
| BEELINE 2020 Figure 4 · GSD | GSD | AUPRC ratio, unitless | 1.26 | 1.37 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 4 · GSD | GSD | Early precision ratio, unitless | 1.45 | 1.42 | GRNBoost2 |
| BEELINE 2020 Figure 4 · GSD | GSD | Early precision ratio (activating edges only), unitless | 1.97 | 2.1 | PPCOR (partial Pearson correlation) |
| BEELINE 2020 Figure 4 · GSD | GSD | Early precision ratio (inhibitory edges only), unitless | 0.402 | 0.201 | GRNBoost2 |
| BEELINE 2020 Figure 4 · VSC | VSC | AUPRC ratio, unitless | 2.68 | 2.14 | GRNBoost2 |
| BEELINE 2020 Figure 4 · VSC | VSC | Early precision ratio, unitless | 2.74 | 1.75 | GRNBoost2 |
| BEELINE 2020 Figure 4 · VSC | VSC | Early precision ratio (inhibitory edges only), unitless | 2.74 | 1.75 | GRNBoost2 |
| BEELINE 2020 Figure 4 · HSC | HSC | AUPRC ratio, unitless | 1.93 | 1.73 | GRNBoost2 |
| BEELINE 2020 Figure 4 · HSC | HSC | Early precision ratio, unitless | 1.95 | 1.63 | GRNBoost2 |
| BEELINE 2020 Figure 4 · HSC | HSC | Early precision ratio (activating edges only), unitless | 2.36 | 2.26 | GRNBoost2 |
| BEELINE 2020 Figure 4 · HSC | HSC | Early precision ratio (inhibitory edges only), unitless | 2.2 | 1.76 | GRNBoost2 |
| BEELINE 2020 Figure 4 · mCAD | mCAD | AUPRC ratio, unitless | 0.771 | 0.772 | Tie |
| BEELINE 2020 Figure 4 · mCAD | mCAD | Early precision ratio, unitless | 0.947 | 0.879 | GRNBoost2 |
| BEELINE 2020 Figure 4 · mCAD | mCAD | Early precision ratio (activating edges only), unitless | 0 | 0 | Tie |
| BEELINE 2020 Figure 4 · mCAD | mCAD | Early precision ratio (inhibitory edges only), unitless | 0.79 | 0.79 | Tie |
What disagrees
- Other studies
- rank GRNBoost2 above PPCOR (partial Pearson correlation) in 47 of 55 comparisons
- This study
- ranks PPCOR (partial Pearson correlation) above GRNBoost2 in 9 of 12 comparisons
- Compared
- GRNBoost2, PPCOR (partial Pearson correlation)
- Study that disagrees
- BEELINE: 10.1038/s41592-019-0690-6 source data published 2020
- Other studies
- BEELINE: 10.1038/s41592-019-0690-6 source data published 2020, BEELINE: 10.1038/s41592-019-0690-6 source data published 2020
- Rule
- Ordering reversal; same task, different protocols
- Model identity
- Matched across studies through reviewed links
The compared numbers
Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.
In the other studies (110 results)
Follow-up
Current state: Not yet reviewed.
No one has reviewed this finding yet.
How to help resolve this
If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.
Finding discrepancy-0311356983b743d6 · All contested results · How findings are made and followed up