Configuration

STELLA, OpenRouter default backend (BioML-bench)

STELLA, OpenRouter default backend: STELLA with 'the OpenRouter default' as LLM backend.

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluations and results

6 evaluations · 23 results. Different protocols are not a single leaderboard.

Sorted by Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Protocol and dataset	Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted)	Leaderboard percentile (mean over tasks in the domain; penalized: runs without a valid submission scored as percentile 0)	Proportion (share of tasks that yield a valid, gradable submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0)	Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); non-penalized: runs without a valid submission omitted)	Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); penalized: runs without a valid submission scored as percentile 0)	Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); non-penalized: runs without a valid submission omitted)	Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0)	Rank (mean relative rank among the five Table 4 agents (Dummy, MLAgentBench, AIDE, STELLA, Biomni) by leaderboard percentile, over tasks in the domain; non-penalized: runs without a valid submission omitted)	Rank (mean relative rank among the five Table 4 agents (Dummy, MLAgentBench, AIDE, STELLA, Biomni) by leaderboard percentile, over tasks in the domain; penalized: runs without a valid submission scored as percentile 0)
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks, BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	61.97 ± 13.91% (best in column)	Not reported	Not reported	46.7 ± 22.6% (best in column)	Not reported	68.3 ± 15.0% (best in column)	Not reported	1.90 ± 0.33	Not reported
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks, BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	34.98 ± 12.51%	Not reported	Not reported	16.7 ± 12.4%	Not reported	45.8 ± 18.7%	Not reported	Not reported	Not reported
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks, BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	28.84 ± 7.84%	Not reported	Not reported	13.9 ± 7.3%	Not reported	25.0 ± 11.0%	Not reported	Not reported	Not reported
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks, BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	Not reported	28.84 ± 7.84%	100% (best in column)	Not reported	13.9 ± 7.3%	Not reported	25.0 ± 11.0%	Not reported	Not reported
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks, BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	Not reported	34.98 ± 12.51%	100% (best in column)	Not reported	16.7 ± 12.4%	Not reported	45.8 ± 18.7%	Not reported	Not reported
STELLA, OpenRouter default backend (BioML-bench)Evaluation	BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks, BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	Not reported	54.23 ± 14.09% (best in column)	85%	Not reported	40.0 ± 20.3% (best in column)	Not reported	60.0 ± 15.0% (best in column)	Not reported	2.30 ± 0.54

All 23 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	25.0 ± 11.0% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 11.0 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (non-penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Drug Discovery', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	13.9 ± 7.3% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 7.3 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (non-penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Drug Discovery', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	28.84 ± 7.84% Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 7.84 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (non-penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Drug Discovery', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	25.0 ± 11.0% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 11.0 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Drug Discovery', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	13.9 ± 7.3% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 7.3 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Drug Discovery', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	100% Proportion (share of tasks that yield a valid, gradable submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Drug Discovery', row 'STELLA', column 'Completion Rate (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench drug discovery domain: agent leaderboard percentile over 9 tasks Dataset subset: BioML-bench v0.1-alpha drug discovery tasks (9 tasks)	28.84 ± 7.84% Leaderboard percentile (mean over tasks in the domain; penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 7.84 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench drug discovery (penalized) lit-agents-20261013-biomlbench-protocol-drug-discovery Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Drug Discovery', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	45.8 ± 18.7% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 18.7 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (non-penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Protein Engineering', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	16.7 ± 12.4% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 12.4 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (non-penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Protein Engineering', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	34.98 ± 12.51% Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 12.51 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (non-penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Protein Engineering', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	45.8 ± 18.7% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 18.7 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Protein Engineering', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	16.7 ± 12.4% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 12.4 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Protein Engineering', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	100% Proportion (share of tasks that yield a valid, gradable submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Protein Engineering', row 'STELLA', column 'Completion Rate (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench protein engineering domain: agent leaderboard percentile over 6 tasks Dataset subset: BioML-bench v0.1-alpha protein engineering tasks (6 tasks)	34.98 ± 12.51% Leaderboard percentile (mean over tasks in the domain; penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 12.51 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench protein engineering (penalized) lit-agents-20261013-biomlbench-protocol-protein-engineering Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Protein Engineering', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	68.3 ± 15.0% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 15.0 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (non-penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	46.7 ± 22.6% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 22.6 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (non-penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	61.97 ± 13.91% Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted) percent · higher Uncertainty: ± 13.91 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (non-penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	1.90 ± 0.33 Rank (mean relative rank among the five Table 4 agents (Dummy, MLAgentBench, AIDE, STELLA, Biomni) by leaderboard percentile, over tasks in the domain; non-penalized: runs without a valid submission omitted) unitless · lower Uncertainty: ± 0.33 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (non-penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'STELLA', column 'Mean Rank'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	60.0 ± 15.0% Proportion (share of tasks where the agent's mean score exceeds the median human leaderboard submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 15.0 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Single Cell Omics', row 'STELLA', column 'Above Median (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	40.0 ± 20.3% Proportion (share of tasks where the agent reaches a public leaderboard medal threshold, MLE-bench definition (Section 3.4); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 20.3 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Single Cell Omics', row 'STELLA', column 'Any Medal (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	85% Proportion (share of tasks that yield a valid, gradable submission (Section 3.4 definition); penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Single Cell Omics', row 'STELLA', column 'Completion Rate (%)'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	54.23 ± 14.09% Leaderboard percentile (mean over tasks in the domain; penalized: runs without a valid submission scored as percentile 0) percent · higher Uncertainty: ± 14.09 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Single Cell Omics', row 'STELLA', column 'Leaderboard Percentile'
Configuration: STELLA, OpenRouter default backend (BioML-bench)	Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)	2.30 ± 0.54 Rank (mean relative rank among the five Table 4 agents (Dummy, MLAgentBench, AIDE, STELLA, Biomni) by leaderboard percentile, over tasks in the domain; penalized: runs without a valid submission scored as percentile 0) unitless · lower Uncertainty: ± 0.54 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM. Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source STELLA on BioML-bench single cell omics (penalized) lit-agents-20261013-biomlbench-protocol-single-cell-omics Aggregation: Not reported BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Penalized', domain 'Single Cell Omics', row 'STELLA', column 'Mean Rank'

Source checking is not independent reproduction. Release 2026-11-02-a7f03c9e55a5.

Use this configuration

How it works, versions and access

Additional methodological details, versions and access terms have not yet been extracted into a structured profile for this configuration. Consult the linked sources and evaluation records below.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statement	Original source and location	Review and provenance
Parameters STELLA with 'the OpenRouter default' as LLM backend (Section 3.3) Individual claims	science-machine/biomlbench repository at commit 673f3d8 Original source ↗ Repository agents/stella/config.yaml at commit 673f3d8 Version: Git commit 673f3d8d3462fc6cdcbbd9695fba612b9332ca65 (2025-09-14, head of main at retrieval); README title 'BioML-bench (v0.1-alpha)' Retrieved: 2026-10-13T12:38:10Z	conflicting claim ["source-hash-verification","independent-cell-check"] · 2026-10-13 Audit details Independent review 2026-10-13: matched the cited repository file at commit 673f3d8. Conflicting source claim: `"Repository default for STELLA: model: claude-sonnet-4, with OpenRouter, Anthropic, Mem0 and Gemini API keys passed; time_limit 57600 seconds"`. Claim record status: source_checked. Field: `attributes.parameters` Claim: lit-agents-20261013-biomlbench-claim-stella-repo-llm Source artifact SHA-256: `c23978275652d61452e710dfdf94613237b85ab9b90605af457d082c351ae559` Hash scope: SHA-256 of the codeload tar.gz archive for the commit; two downloads gave identical bytes. Per-file hashes of the files cited are in sources.md. Inspected artifact Extraction artifact SHA-256: `c23978275652d61452e710dfdf94613237b85ab9b90605af457d082c351ae559` Extraction artifact

How to read the evidence

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

1 source record and release history

BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Original source · bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-agents-20261013-biomlbench-config-stella-openrouter-default

areas: molecular-interactions; protein-function; single-cell
contexts: research
method types: foundation_model
reported name: STELLA
parameters: STELLA with 'the OpenRouter default' as LLM backend (Section 3.3)
source locator: Section 3.3
access reuse: Open-source agent; LLM backend is a hosted API
missing metadata: model: reason: conflicting; note: The paper says STELLA used 'the OpenRouter default' LLM and names no model. The repository config at commit 673f3d8 sets model: claude-sonnet-4 and passes OpenRouter, Anthropic, Gemini and Mem0 keys. No uses_model link is made.; version: reason: unreported; note: No release or commit of STELLA is printed
limitations: Run by the BioML-bench authors in the configuration described in Section 3.3; four replicate runs per task.
provenance note: Agent configuration as run in BioML-bench (bioRxiv v3).

Related records

configuration of: STELLA
subject: config-stella-openrouter-default: attributes.parameters
system: STELLA on BioML-bench drug discovery (non-penalized)
system: STELLA on BioML-bench drug discovery (penalized)
system: STELLA on BioML-bench protein engineering (non-penalized)
system: STELLA on BioML-bench protein engineering (penalized)
system: STELLA on BioML-bench single cell omics (non-penalized)
system: STELLA on BioML-bench single cell omics (penalized)

Suggest a correction