Benchmark

PEtab benchmark collection

The PEtab collection supports evaluation of computational methods for fitting mathematical models to observations.

Sources

Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table

No reviewed evaluations are linked here in this release. See the sources and separately identified configurations below.

0 evaluations · 0 results

Overview

Datasets

Individual model/measurement problems in PEtab format, with problem-specific observables and noise assumptions.

Metrics

Objectives depend on each PEtab measurement/noise model and the selected optimization-performance criterion; there is no universal prediction metric.

inapplicable

Allowed inputs

PEtab model, parameter, condition, observable and measurement tables.

Sources

Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table

Evaluation procedure diagram

How it worksEvaluation procedure

Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.

Sources

Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table

Source reviewed · Automated source review, 2026-09-16. All specifications and missing details

Results

All evaluations

0 evaluations · 0 results. Different protocols are not a single leaderboard.

Applied filters: All linked evaluations

No evaluations linked in this release.

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

The collection packages dynamical biological models with the experimental measurements and observation/noise assumptions needed for parameter estimation. A benchmark run specifies a model, solver and inference procedure against this fixed problem. Calibration fit, computational reliability and predictive validation are different assessment targets.

Sources

petab primary benchmark evidence · Sections 2.2–2.4: observations, noise models and experimental conditions

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

No concrete protocols are explicitly linked to this suite. Protocol identification and baseline selection are outstanding.

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run this benchmark

The repository provides PEtab problems and links INSTALL.md. It is a benchmark-problem collection rather than a prescribed optimizer run; choose a model, solver, initialization and budget separately. Source-model/data licenses can differ from the repository license.

A maintained rewire runner has not been verified for this benchmark. Check data access, weights, licences, dependencies and hardware in the linked official documentation; requirements have not been fully extracted.

Benchmarking-Initiative/Benchmark-Models-PEtab / README.md · README.md lines 7–12 and 54–68 (Overview, license, installation and version citation)

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths supported by sources

Standardized problem definitions expose differences in objectives and measurement-error assumptions.
Sources
Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table

Limitations and conditions

The original collection paper describes the scientific problem format; the current PEtab repository is a later versioned distribution. Fitting calibration data is not evidence of accuracy on an independent experimental condition.
Sources
petab primary benchmark evidence · Sections 2.2–2.4: observations, noise models and experimental conditions

Profile review details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Stable record: discovery-benchmark-petab-benchmark-collection

Specifications

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Datasets	Individual model/measurement problems in PEtab format, with problem-specific observables and noise assumptions. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Splits	A supervised train/test split is not intrinsic to the parameter-estimation problem collection; the chosen study must define any held-out observations. · Not applicable Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Metrics	Objectives depend on each PEtab measurement/noise model and the selected optimization-performance criterion; there is no universal prediction metric. · Not applicable Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Baselines	The collection provides common problem definitions for comparing modeling/estimation methods, not a fixed universal baseline. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Leakage controls	The collection provides mechanistic models with calibration data, observation functions and experimental conditions. It is intended to compare numerical inference methods; a predictive train/test exclusion policy must be defined by the study using each model. · Not applicable Sources petab primary benchmark evidence · Sections 2.2–2.4: observations, noise models and experimental conditions
Uncertainty	Measurement errors can be fixed from experiments or estimated jointly through explicit noise models. Parameter uncertainty and optimizer variability are different quantities and require their own evaluation procedure. Sources petab primary benchmark evidence · Sections 2.2–2.4: observations, noise models and experimental conditions
Entity type	Collection of parameter-estimation benchmark problems. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Organisms	Organism identity is problem-specific; the collection spans distinct systems. · Not applicable Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Assays	Problem-specific observations with explicit measurement/noise models. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Allowed inputs	PEtab model, parameter, condition, observable and measurement tables. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table
Adaptation	Numerical parameter estimation against supplied observations; optimizer settings define the tested method. Sources Benchmarking-Initiative/Benchmark-Models-PEtab official source · Pinned README: collection description; benchmark problem table

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resource	Version	Reference
Benchmark problems for dynamic modeling of intracellular processes	PMC6735869	Read source DOI: 10.1093/bioinformatics/btz020

Historical gaps recorded on 2026-09-17

Collection of parameter-estimation problems, not one universal biological accuracy benchmark. Model/dataset identifiers, objective, solver and restart budget are necessary before comparing runs. No universal score generated.

Search and extraction details

source found structured extraction pending

Searches

PEtab benchmark collection primary paper benchmark results

Evidence locations

Table1 model collection; benchmark problem definition and optimisation results

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

18 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Diagram steps Allowed inputs: PEtab model, parameter, condition, observable and measurement tables. Splits: A supervised train/test split is not intrinsic to the parameter-estimation problem collection; the chosen study must define any held-out observations. Metrics: Objectives depend on each PEtab measurement/noise model and the selected optimization-performance criterion; there is no universal prediction metric. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Diagram title Evaluation procedure Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Datasets Individual model/measurement problems in PEtab format, with problem-specific observables and noise assumptions. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Splits A supervised train/test split is not intrinsic to the parameter-estimation problem collection; the chosen study must define any held-out observations. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	inapplicable automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Adaptation Numerical parameter estimation against supplied observations; optimizer settings define the tested method. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.10.value` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Metrics Objectives depend on each PEtab measurement/noise model and the selected optimization-performance criterion; there is no universal prediction metric. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	inapplicable automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Baselines The collection provides common problem definitions for comparing modeling/estimation methods, not a fixed universal baseline. Individual claims	Benchmarking-Initiative/Benchmark-Models-PEtab official source Original source ↗ Pinned README: collection description; benchmark problem table Version: ddaa86d13f708926c57ec8918ce75a6b50e2e562 Retrieved: 2026-09-16T10:30:24.477301+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.3.value` Source artifact SHA-256: `5a089ca429ed2a314e257fddacd70251c863e62a3547b793c38f56785861cac1` Hash scope: Hash scope not separately documented; inspect source record
Leakage controls The collection provides mechanistic models with calibration data, observation functions and experimental conditions. It is intended to compare numerical inference methods; a predictive train/test exclusion policy must be defined by the study using each model. Individual claims	petab primary benchmark evidence Original source ↗ Sections 2.2–2.4: observations, noise models and experimental conditions Version: PMC6735869 Retrieved: 2026-09-16T21:04:56.573248+00:00	inapplicable automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.4.value` Source artifact SHA-256: `9a9f73410331cb6d0ee148f6872f191393cbaa334f3437ed7fd0fe289476e7a7` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Uncertainty Measurement errors can be fixed from experiments or estimated jointly through explicit noise models. Parameter uncertainty and optimizer variability are different quantities and require their own evaluation procedure. Individual claims	petab primary benchmark evidence Original source ↗ Sections 2.2–2.4: observations, noise models and experimental conditions Version: PMC6735869 Retrieved: 2026-09-16T21:04:56.573248+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.5.value` Source artifact SHA-256: `9a9f73410331cb6d0ee148f6872f191393cbaa334f3437ed7fd0fe289476e7a7` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: discovered

4 source records and release history

Benchmarking-Initiative/Benchmark-Models-PEtab official source · Original source · ddaa86d13f708926c57ec8918ce75a6b50e2e562
Benchmark problems for dynamic modeling of intracellular processes · Original source · PMC6735869
Benchmarking-Initiative/Benchmark-Models-PEtab / README.md · Original source · Git commit ddaa86d13f708926c57ec8918ce75a6b50e2e562
petab primary benchmark evidence · Original source · PMC6735869

Download this release

Technical metadata and extraction receipts

Stable ID: discovery-benchmark-petab-benchmark-collection

areas: mechanistic-biology
entity level: suite
scope note: Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
task: Parameter estimation for biological dynamical models
version: Not reported
benchmark research: review date: 2026-09-17; status: source_found_structured_extraction_pending; primary sources: evidence-expansion-p2-evidence-discovery-final-petab-9a9f73410331; inspected locators: Table1 model collection; benchmark problem definition and optimisation results; searched queries: PEtab benchmark collection primary paper benchmark results; gaps: Collection of parameter-estimation problems, not one universal biological accuracy benchmark. Model/dataset identifiers, objective, solver and restart budget are necessary before comparing runs. No universal score generated.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
historical missing metadata: dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
entity classification: review date: 2026-09-17; rationale: The cited profile describes a collection of evaluation tasks or protocols; retain it as the top-level benchmark suite. Its datasets and individual protocols remain separate records.; source ids: src-discovery-benchmarking-initiative-benchmark-models-petab; source locator: Pinned README: collection description; benchmark problem table; ambiguities: None recorded
run documentation: record id: discovery-benchmark-petab-benchmark-collection; source ids: run-doc-petab-benchmark-collection-readme-md-ddaa86d1; status: official_documentation_linked; summary: The repository provides PEtab problems and links INSTALL.md. It is a benchmark-problem collection rather than a prescribed optimizer run; choose a model, solver, initialization and budget separately. Source-model/data licenses can differ from the repository license.; source locator: README.md lines 7–12 and 54–68 (Overview, license, installation and version citation)

Related records

Suggest a correction