Demo history · DataPilot and ReturnPilot runs marked “simulated” come from af-run demo — simulated agent behaviour scored by the real graders, statistics and gate; not measurements of the live apps.

opt-returnpilot-2026-09-13 · AgentForge

finishedReturnPilotparent v2

opt-returnpilot-2026-09-13

10 candidates over 2 nights · components prompts.system, tool_descriptions.check_return_eligibility, tool_descriptions.issue_refund, few_shots

target-agent call budget711 / 1,500 · 47%

minibatch: 10 train cases (½ parent failures)
nightly slice: 300 calls
max iterations · patience: 12 · 8
started: Sep 13, 07:55 UTC
finished: last month

Candidate tree

minibatch Δ vs parent · ring = Pareto front

gated / promotedevaluatedon Pareto frontrejected by editor

Pareto front

val pass rate vs list-price cost / case

prompt candidate on frontconfig-search point (§9.3)

candidate c1 · iteration 1 · component

prompts.system

“adds: - Before issue_refund, always call check_return_eligibility for the same item an”

evaluated on valPareto frontval 69%$0.0019 / case

Editor checks · §9.4

schema
valid profile.v1
locked_keys
only optimizable keys changed
keep_blocks
all <keep> blocks intact
length_cap
within 1.3× + 400 chars
case_literals
no batch literals copied
deny_list
no weakening phrases
non_empty_diff
changed: prompts.system
param_ranges
params within declared ranges

minibatch 8/10 → 9/10 on the same 10 train cases

Diff vs parent seed

prompts.system2185 → 2397 chars

⋯ 2,025 unchanged chars ⋯customer (from earlier conversations). When they say "my usual" or "like last time", answer from these memories directly — don't ask again: {memories} {summary} - Before issue_refund, always call check_return_eligibility for the same item and refund at most its max_refund. - Offer a ticket when no tool can help (cancellations, disputes) instead of improvising an action.

Best candidate c1 → promotion gate

changed prompts.system · selected by highest val mean (ties → lower cost)

Not sent to the gate yet. The gate runs the TEST split × pass^2 and the red-team core for this candidate and the active profile.

Iteration log

parent → component → child · minibatch scores on the same 10 train cases

Iter	Parent	Component	Child	Minibatch	Outcome
1	seed	prompts.system	c1	8 → 9	to val
2	c1	tool_descriptions.check_return_eligibility	c2	7 → 7	rejected
3	c1	tool_descriptions.issue_refund	c3	6 → null	rejected
4	c1	few_shots	c4	6 → 6	rejected
5	c1