Demo history · DataPilot and ReturnPilot runs marked “simulated” come from af-run demo — simulated agent behaviour scored by the real graders, statistics and gate; not measurements of the live apps.

opt-datapilot-2026-09-18 · AgentForge

finishedDataPilotparent v2

opt-datapilot-2026-09-18

11 candidates over 2 nights · components prompts.sql_direct, prompts.planner, few_shots

target-agent call budget2,107 / 4,000 · 53%

minibatch: 10 train cases (½ parent failures)
nightly slice: 1000 calls
max iterations · patience: 10 · 6
started: Sep 18, 07:55 UTC
finished: last month

Candidate tree

minibatch Δ vs parent · ring = Pareto front

gated / promotedevaluatedon Pareto frontrejected by editor

Pareto front

val pass rate vs list-price cost / case

prompt candidate on frontconfig-search point (§9.3)

candidate c4 · iteration 4 · component

prompts.sql_direct

“adds: - Return exactly the columns the question names, in that order.”

promotedPareto frontval 82%$0.0023 / case

Editor checks · §9.4

schema
valid profile.v1
locked_keys
only optimizable keys changed
keep_blocks
all <keep> blocks intact
length_cap
within 1.3× + 400 chars
case_literals
no batch literals copied
deny_list
no weakening phrases
non_empty_diff
changed: prompts.sql_direct
param_ranges
params within declared ranges

minibatch 4/10 → 6/10 on the same 10 train cases

Diff vs parent c1

prompts.sql_direct1229 → 1293 chars

⋯ 1,069 unchanged chars ⋯ply with the SQL in one ```sql fenced block and nothing else. - Compute ratios and percentages in one expression with CAST(... AS REAL); return only that value. - Return exactly the columns the question names, in that order.

Best candidate c4 → promotion gate

changed prompts.sql_direct · selected by highest val mean (ties → lower cost)

Test split · n=60

55% → 72%

gain +17 pts, 95% CI +7 to +28

Paired test

p=0.006

McNemar exact · b=1 c=11

Verdict

quality

n=60: 80% power only for ≥17 pts · P(detect +10)=43%

Hard safety
0 must_not, 0 canary on test + red-team

Iteration log

parent → component → child · minibatch scores on the same 10 train cases

Iter	Parent	Component	Child	Minibatch	Outcome
1	seed	prompts.sql_direct	c1	9 → 10	to val
2	c1	prompts.planner	c2	8 → 8	rejected
3	c1	few_shots	c3	6 → 6	rejected
4	c1	prompts.sql_direct	c4	4 → 6	to val
5	c4