In this commit, we act on two independent advisor reviews that
reframed the program: the paradigm ceiling we have been attributing to
algorithm space is partly a measurement ceiling, and the validation
story has holes that would surface immediately upstream.
Measurement: the evaluator now emits separate objective axes
(success, retry efficiency with shards disentangled from retries, and
fee efficiency) so the engine's hybrid Pareto frontier can keep
specialists alive, and evaluation caching is enabled now that the
evaluator is verified deterministic. The split corpus generator gains
--split-leads, replacing the single ambitious payment -- which left
two thirds of every file's score as free probes and quantized
minibatch selection above the very signal being selected for -- with
a descending ladder of mandatory-split payments whose completion count
grades the score. The original --split output is regression-tested
byte-identical.
Validation: sweep_validate.py replaces ad-hoc sweeps with paired
per-file comparisons, bootstrap confidence intervals, and sign tests;
gen_mainnet_scenarios.py generates multi-vantage mainnet corpora with
log-spaced source degrees (2024 down to 2) so claims stop resting on a
single hub-resident vantage; and params_lnd_bimodal.json adds the
baseline arm reviewers will ask for first, since lnd ships a bimodal
estimator that our defaults-only comparisons never exercised. The
exp-010 writeup gains a pre-registered caveat, logged before the live
runs finish, that corpus resolution may mute their verdicts.
In this commit, we add the Python harness that drives GEPA's
optimize_anything over the routesim evaluator. Two modes are supported:
parameter mode tunes the existing heuristic's knobs as a JSON candidate,
while code mode evolves the entire routing algorithm as the full Go
source of candidate_impl.go, compiled per-eval via go build -overlay
with compiler errors fed back to the proposer as reflection signal.
The evaluator scores success rate with small saturating penalties for
retry attempts and fees, and guards against reward hacking by rejecting
candidates that reach for unsafe, reflect, or exec surfaces. The
reflection LM runs through the Codex CLI in headless mode via a small
LM-protocol wrapper, and omni-style two-phase composition (parallel
explore, then a fresh engine seeded with the winner) is available
alongside plain and adaptive runs. The background prompt encodes the
insights discovered by prior champion runs so follow-up evolution builds
on them rather than rediscovering them.
