BEELINE

Overview

Datasets

Single-cell expression inputs and ground-truth regulatory networks, configured per dataset.

Metrics

AUROC, AUPRC, early-precision ratio, signed early precision, rank correlation and top-edge overlap; runtime/network diagnostics are separate outputs.

Allowed inputs

Expression matrix, gene metadata and ground-truth edges for evaluation.

Sources

Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results

Evaluation procedure diagram

How it worksEvaluation procedure

Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.

Sources (2)

Murali-group/Beeline official source; beeline primary benchmark evidence · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets

Source reviewed · Automated source review, 2026-09-16. All specifications and missing details

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

Find a protocol, dataset or metric

Evaluation scope and datasetMetric and reported setup

BEELINE 2020 Figure 5 · mESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500: Early Precision Ratio

Early Precision Ratio (ratio) · Higher values are better.

BEELINE 2020 Figure 5 · mESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · mESC

Evidence origin: Author-reported evaluation.

BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 · Figure 5 source CSV row 17, column 6 through Figure 5 source CSV row 17, column 11

Missing source cells and quarantined conflicts are recorded in acquisition and audit tables. Per-result scoring denominators may be unreported.

Comparison details and limitations

Complete selected source table is retained across source-order panels. These point estimates do not establish statistical significance or a universal ranking.

Source-specific evaluation. Input conditions: Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500. No equivalence to other releases, protocols or model families is inferred.
Exact source-defined evaluation scope; reported scores are not rewire reproductions.

Automated source review: 2026-09-19. Numerical source review does not establish independent reproduction.

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Find a tested model or method

Showing 6 of 6 matching rows.

Zoom to observed range

Tested configuration

0.9951.021.051.071.1

Reported score

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

BEELINE evaluates gene regulatory network inference from single-cell expression. It runs inference algorithms on synthetic, curated-model and experimental datasets, then compares ranked predicted edges with known or constructed reference networks. Pseudotime requirements and reference-network reliability are part of each evaluation.

Sources

beeline primary benchmark evidence · Methods: algorithm execution, simulated/curated datasets and experimental datasets

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Protocols

These source-backed links do not make different protocols or scores interchangeable.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 108 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

Baseline status by linked protocol

BEELINE 2020 Figure 5 · mESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: log/gof; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 2 · LI · 0/2 roles measured
BEELINE 2020 Figure 2 · TF · 0/2 roles measured
BEELINE 2020 Figure 2 · CY · 0/2 roles measured
BEELINE 2020 Figure 4 · GSD · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 4 · VSC · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 2 · BFC · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: log/gof; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 4 · HSC · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: STRING; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 2 · LL · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · hHep · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · hESC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mDC · Reference network: STRING; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-E · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured
BEELINE 2020 Figure 5 · MHSC-GM · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 4 · mCAD · 0/2 roles measured
BEELINE 2020 Figure 5 · mHSC-L · Reference network: Cell-type specific ChIP-Seq; Gene selection: TFs+500 · 0/2 roles measured
BEELINE 2020 Figure 2 · BF · 0/2 roles measured
BEELINE 2020 Figure 5 · mESC · Reference network: Non-specific ChIP-Seq; Gene selection: TFs+1000 · 0/2 roles measured

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run this benchmark

Choose a concrete protocol before running an evaluation. Its inputs, split and scoring rules determine which results can be compared.

Official run instructions

Run and evaluate the supplied BEELINE GSD example configuration, then generate AUPRC/AUROC plots.

Checked against the official instructions on 2026-09-17. These commands have not been executed by rewire. Running them does not automatically reproduce the published scores.

Before you start

Git, conda and a functioning Docker installation; the repository executes supported algorithms in containers.
The supplied config-files/config.yaml expects GSD expression, pseudotime and ground-truth files under inputs/example/GSD. Verify the local data layout before running.

1. Check out the reviewed repository
Repository checkout wrapper: the detached revision selects the exact official source inspected for this guide.
```
git clone https://github.com/Murali-group/Beeline.git
cd Beeline
git checkout --detach 37464085eb8a95d6cc6a3d3a3c649d36db6052ed
```
Murali-group/Beeline / README.md · Pinned repository revision; README.md
2. Set up the environment and algorithm images
These setup commands use the official default image-pull behavior. Conda must already be initialized in the shell; the README illustrates a Miniconda-specific initialization path.
```
bash utils/setupAnacondaVENV.sh
bash utils/initialize.sh
conda activate BEELINE
```
Murali-group/Beeline / README.md · README.md lines 9–36
3. Run the configured algorithms
This executes the algorithms and datasets enabled by the YAML file. The README points to a missing VSC file; this guide explicitly substitutes the supplied GSD example config, so it is a different example dataset.
```
python BLRunner.py -c config-files/config.yaml
```
Murali-group/Beeline / README.md; Murali-group/Beeline / config-files/config.yaml · README.md lines 40–52; config-files/config.yaml (input_settings/datasets and output_settings)
4. Evaluate and plot ROC/precision-recall performance
The flags are selected from the official tables: evaluator -a computes AUPRC/AUROC; plotter -a and -r produce their corresponding figures. The README points to a missing VSC file; this guide explicitly substitutes the supplied GSD example config, so it is a different example dataset.
```
python BLEvaluator.py -c config-files/config.yaml -a
python BLPlotter.py -c config-files/config.yaml -o ./plots -a -r
```
Murali-group/Beeline / README.md; Murali-group/Beeline / config-files/config.yaml · README.md lines 54–85; config-files/config.yaml (input_settings/datasets and output_settings)

Expected outputs

Ranked edges under outputs/example_run_GSD/GSD/<run_id>/<algorithm_id>/rankedEdges.csv, using output_settings in the supplied config.
AUPRC and AUROC plot files under ./plots in PDF and PNG formats.

Scope and limitations

The README VSC command references a configuration absent from the pinned repository. This guide uses the checked-in GSD example instead and does not claim VSC results.
The Docker tags are not image-digest pinned by the README; this code revision alone does not freeze algorithm containers.
Ground-truth network availability and matching input/configuration paths are prerequisites; no data-download command is invented.
The inspected instructions do not establish a minimum RAM/VRAM requirement, wall-clock runtime or monetary cost; none is inferred.

Specifications

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Datasets	Single-cell expression inputs and ground-truth regulatory networks, configured per dataset. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Splits	BEELINE infers a network from each expression dataset and scores predicted edges against a reference network. Simulated replicates and experimental contexts are separate benchmark cases; the benchmark is not a single supervised train/validation/test classification split. Sources beeline primary benchmark evidence · Methods: algorithm execution, simulated/curated datasets and experimental datasets
Metrics	AUROC, AUPRC, early-precision ratio, signed early precision, rank correlation and top-edge overlap; runtime/network diagnostics are separate outputs. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Baselines	Containerized inference methods share ranked-edge outputs for a common evaluator. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Leakage controls	Reference networks are used to score inferred edges, while inputs are expression and, for some methods, pseudotime. Algorithm assumptions, simulated ground truth and experimental reference construction are explicit; no single held-out-gene training protocol applies to all methods. Sources beeline primary benchmark evidence · Methods: algorithm execution, simulated/curated datasets and experimental datasets
Uncertainty	Plotting supports multiple-run distributions, but a particular repeated-run design is not fixed by the README. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Entity type	Gene-regulatory network inference evaluation pipeline. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Organisms	Human transcription-factor metadata is supported; the chosen single-cell dataset defines organism scope. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Assays	Single-cell expression with a reference regulatory network. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Allowed inputs	Expression matrix, gene metadata and ground-truth edges for evaluation. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results
Adaptation	Inference algorithms operate on the supplied expression data; labeled reference edges are used for assessment. Sources Murali-group/Beeline official source · Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

Paper or primary resource	Version	Reference
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data	PMC7098173	Read source DOI: 10.1038/s41592-019-0690-6

Historical gaps recorded on 2026-09-17

The catalogue now holds 588 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

Primary XML has no numeric table-wrap results; comparisons are figures and supplementary data. Exact full model-by-dataset scores require supplemental/source-data extraction, not digitizing figure heights.

Search and extraction details

primary figures located

Searches

"Normalized compression distance for DNA classification"
"AUPRC" "in-silico perturbation"
"mouse" "Geneformer" "PMC11964219"
"antibody flexibility" "PMC12530544"
"scXDR" cross dataset drug response
"Hi-enhancer" 2025
"PlantCaduceus2" 2025
"COBRA" "RNA" "PMC12790621"
"PMC12425018"
"PMC12758598"
"DeepInterAware" 2025
"BarcodeBERT" unseen species
"PMC8207588"
"PMC10723403"
"PMC11167433"
"PMC12826486"
"PMC11565894"
"PMC11695915"
"PMC9178954"
"PMC13418759"
"PMC12516880"
"PMC11815853"
"PMC12417085"
"PMC12889687"
"PMC12635123"
"PMC12957212"
"PMC12453675"
"PMC12493982"
"PMC12798952"
"PMC13132462"
"PMC11785235"
"PMC8763943"
"PMC13182013"
"PMC12619997"
ProkBERT promoter benchmark
scPertEval benchmark paper
GlycanML benchmark 2405.16206
AMBER metagenome binning assessment 2018 PMC6022608
BEELINE gene regulatory network benchmark 2020 Pratapa
FLIP benchmark protein fitness landscape inference 2021
Genomic Benchmarks collection genomic sequence classification PMC10150520
mRNABench PMC12265608
PFMBench 2506.14796
ProteinBench 2409.06744

Evidence locations

Results and figures
Online Methods

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

27 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims	beeline primary benchmark evidence Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: PMC7098173 Retrieved: 2026-09-16T21:04:55.226418+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `99b59a6941878779b34ab8eed9ea27c8d8967c24fb67b92fce418e7f512d3a11` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record
Diagram steps Allowed inputs: Expression matrix, gene metadata and ground-truth edges for evaluation. Splits: BEELINE infers a network from each expression dataset and scores predicted edges against a reference network. Simulated replicates and experimental contexts are separate benchmark cases; the benchmark is not a single supervised train/validation/test classification split. Metrics: AUROC, AUPRC, early-precision ratio, signed early precision, rank correlation and top-edge overlap; runtime/network diagnostics are separate outputs. Individual claims	beeline primary benchmark evidence Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: PMC7098173 Retrieved: 2026-09-16T21:04:55.226418+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `99b59a6941878779b34ab8eed9ea27c8d8967c24fb67b92fce418e7f512d3a11` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram steps Allowed inputs: Expression matrix, gene metadata and ground-truth edges for evaluation. Splits: BEELINE infers a network from each expression dataset and scores predicted edges against a reference network. Simulated replicates and experimental contexts are separate benchmark cases; the benchmark is not a single supervised train/validation/test classification split. Metrics: AUROC, AUPRC, early-precision ratio, signed early precision, rank correlation and top-edge overlap; runtime/network diagnostics are separate outputs. Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record
Diagram title Evaluation procedure Individual claims	beeline primary benchmark evidence Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: PMC7098173 Retrieved: 2026-09-16T21:04:55.226418+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `99b59a6941878779b34ab8eed9ea27c8d8967c24fb67b92fce418e7f512d3a11` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram title Evaluation procedure Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; Methods: algorithm execution, simulated/curated datasets and experimental datasets Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record
Datasets Single-cell expression inputs and ground-truth regulatory networks, configured per dataset. Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record
Splits BEELINE infers a network from each expression dataset and scores predicted edges against a reference network. Simulated replicates and experimental contexts are separate benchmark cases; the benchmark is not a single supervised train/validation/test classification split. Individual claims	beeline primary benchmark evidence Original source ↗ Methods: algorithm execution, simulated/curated datasets and experimental datasets Version: PMC7098173 Retrieved: 2026-09-16T21:04:55.226418+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `99b59a6941878779b34ab8eed9ea27c8d8967c24fb67b92fce418e7f512d3a11` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Adaptation Inference algorithms operate on the supplied expression data; labeled reference edges are used for assessment. Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.10.value` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record
Metrics AUROC, AUPRC, early-precision ratio, signed early precision, rank correlation and top-edge overlap; runtime/network diagnostics are separate outputs. Individual claims	Murali-group/Beeline official source Original source ↗ Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results Version: 37464085eb8a95d6cc6a3d3a3c649d36db6052ed Retrieved: 2026-09-16T10:30:24.835167+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `b9e620179f6b9a9aafd9eaf8b874b8f1fa2c8c4ffdced39f2e075391ec3d476d` Hash scope: Hash scope not separately documented; inspect source record

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: discovered

5 source records and release history

Murali-group/Beeline official source · Original source · 37464085eb8a95d6cc6a3d3a3c649d36db6052ed
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data · Original source · PMC7098173
Murali-group/Beeline / README.md · Original source · Git commit 37464085eb8a95d6cc6a3d3a3c649d36db6052ed
Murali-group/Beeline / config-files/config.yaml · Original source · Git commit 37464085eb8a95d6cc6a3d3a3c649d36db6052ed
beeline primary benchmark evidence · Original source · PMC7098173

Download this release

Technical metadata and extraction receipts

Stable ID: discovery-benchmark-beeline

areas: biological-networks
entity level: suite
scope note: Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
task: Gene regulatory network inference
version: Not reported
benchmark research: review date: 2026-09-17; status: primary_figures_located; primary sources: expansion-p3-beeline; inspected locators: Results and figures; Online Methods; searched queries: "Normalized compression distance for DNA classification"; "AUPRC" "in-silico perturbation"; "mouse" "Geneformer" "PMC11964219"; "antibody flexibility" "PMC12530544"; "scXDR" cross dataset drug response; "Hi-enhancer" 2025; "PlantCaduceus2" 2025; "COBRA" "RNA" "PMC12790621"; "PMC12425018"; "PMC12758598"; "DeepInterAware" 2025; "BarcodeBERT" unseen species; "PMC8207588"; "PMC10723403"; "PMC11167433"; "PMC12826486"; "PMC11565894"; "PMC11695915"; "PMC9178954"; "PMC13418759"; "PMC12516880"; "PMC11815853"; "PMC12417085"; "PMC12889687"; "PMC12635123"; "PMC12957212"; "PMC12453675"; "PMC12493982"; "PMC12798952"; "PMC13132462"; "PMC11785235"; "PMC8763943"; "PMC13182013"; "PMC12619997"; ProkBERT promoter benchmark; scPertEval benchmark paper; GlycanML benchmark 2405.16206; AMBER metagenome binning assessment 2018 PMC6022608; BEELINE gene regulatory network benchmark 2020 Pratapa; FLIP benchmark protein fitness landscape inference 2021; Genomic Benchmarks collection genomic sequence classification PMC10150520; mRNABench PMC12265608; PFMBench 2506.14796; ProteinBench 2409.06744; gaps: Primary XML has no numeric table-wrap results; comparisons are figures and supplementary data. Exact full model-by-dataset scores require supplemental/source-data extraction, not digitizing figure heights.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
historical missing metadata: dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
entity classification: review date: 2026-09-17; rationale: The cited profile describes a collection of evaluation tasks or protocols; retain it as the top-level benchmark suite. Its datasets and individual protocols remain separate records.; source ids: src-discovery-murali-group-beeline; source locator: Pinned README: Overview; Running algorithms; Evaluate results metric table; Plot results; ambiguities: None recorded
run guide: record id: discovery-benchmark-beeline; summary: Run and evaluate the supplied BEELINE GSD example configuration, then generate AUPRC/AUROC plots.; status: source_reviewed_not_executed; prerequisites: Git, conda and a functioning Docker installation; the repository executes supported algorithms in containers.; The supplied config-files/config.yaml expects GSD expression, pseudotime and ground-truth files under inputs/example/GSD. Verify the local data layout before running.; steps: title: Check out the reviewed repository; shell: git clone https://github.com/Murali-group/Beeline.git cd Beeline git checkout --detach 37464085eb8a95d6cc6a3d3a3c649d36db6052ed; explanation: Repository checkout wrapper: the detached revision selects the exact official source inspected for this guide.; source ids: run-doc-beeline-readme-md-37464085; source locator: Pinned repository revision; README.md; title: Set up the environment and algorithm images; shell: bash utils/setupAnacondaVENV.sh bash utils/initialize.sh conda activate BEELINE; explanation: These setup commands use the official default image-pull behavior. Conda must already be initialized in the shell; the README illustrates a Miniconda-specific initialization path.; source ids: run-doc-beeline-readme-md-37464085; source locator: README.md lines 9–36; title: Run the configured algorithms; shell: python BLRunner.py -c config-files/config.yaml; explanation: This executes the algorithms and datasets enabled by the YAML file. The README points to a missing VSC file; this guide explicitly substitutes the supplied GSD example config, so it is a different example dataset.; source ids: run-doc-beeline-readme-md-37464085; run-doc-beeline-config-yaml-37464085; source locator: README.md lines 40–52; config-files/config.yaml (input_settings/datasets and output_settings); title: Evaluate and plot ROC/precision-recall performance; shell: python BLEvaluator.py -c config-files/config.yaml -a python BLPlotter.py -c config-files/config.yaml -o ./plots -a -r; explanation: The flags are selected from the official tables: evaluator -a computes AUPRC/AUROC; plotter -a and -r produce their corresponding figures. The README points to a missing VSC file; this guide explicitly substitutes the supplied GSD example config, so it is a different example dataset.; source ids: run-doc-beeline-readme-md-37464085; run-doc-beeline-config-yaml-37464085; source locator: README.md lines 54–85; config-files/config.yaml (input_settings/datasets and output_settings); outputs: Ranked edges under outputs/example_run_GSD/GSD/<run_id>/<algorithm_id>/rankedEdges.csv, using output_settings in the supplied config.; AUPRC and AUROC plot files under ./plots in PDF and PNG formats.; limitations: The README VSC command references a configuration absent from the pinned repository. This guide uses the checked-in GSD example instead and does not claim VSC results.; The Docker tags are not image-digest pinned by the README; this code revision alone does not freeze algorithm containers.; Ground-truth network availability and matching input/configuration paths are prerequisites; no data-download command is invented.; The inspected instructions do not establish a minimum RAM/VRAM requirement, wall-clock runtime or monetary cost; none is inferred.; source ids: run-doc-beeline-readme-md-37464085; run-doc-beeline-config-yaml-37464085; review: method: official_repository_review; date: 2026-09-17
run documentation: record id: discovery-benchmark-beeline; source ids: run-doc-beeline-readme-md-37464085; run-doc-beeline-config-yaml-37464085; status: source_reviewed_not_executed; summary: Run and evaluate the supplied BEELINE GSD example configuration, then generate AUPRC/AUROC plots. The README VSC command references a configuration absent from the pinned repository. This guide uses the checked-in GSD example instead and does not claim VSC results.; source locator: README.md Setup and Usage; config-files/config.yaml input_settings and output_settings

Related records