lnd/simulation
Olaoluwa Osuntokun d9fbba1938 simulation/lab: WHY.md's attribution caveat gets its answer
In this commit, we resolve the precision-paradise section with
exp-019's measurement: the degraded-attribution run WHY.md said the
8.6x was waiting on has now happened, the champions' margins hold or
widen under unreadable errors, and the number that dies is the 8.6x
itself, replaced by the success-edge claim.
2026-07-27 01:06:23 -07:00
..
champions simulation: reconcile the stale stubs the docs pass found 2026-07-26 14:41:37 -07:00
command-center command-center: resolve the adjudication labels, mx_c3 defends (v59) 2026-07-27 00:53:17 -07:00
lab simulation/lab: WHY.md's attribution caveat gets its answer 2026-07-27 01:06:23 -07:00
.gitignore simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
claude_lm.py simulation: isolate claude -p reflections from user-level hooks 2026-07-25 13:01:36 -07:00
codex_lm.py simulation: run codex searcher agents at xhigh reasoning effort 2026-07-27 00:07:29 -07:00
evaluate.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
evaluate_code.py simulation: recalibrate the abandonment hint 2026-07-27 00:21:54 -07:00
export_run.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
gen_family_corpora.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_mainnet_scenarios.py simulation: adopt advisor corrections to measurement and validation 2026-07-25 02:58:52 -07:00
gen_mainnet_variants.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_scenarios.py routing+routesim: degrade the failure attribution channel 2026-07-27 00:54:37 -07:00
gen_served_weights.py simulation/lab: exp-016, free knowledge helps the champions and hurts lnd 2026-07-26 23:06:50 -07:00
gen_warmup_scenarios.py routing+routesim: separate what a warmup teaches from what it spends 2026-07-26 02:38:57 -07:00
params_lnd_bimodal.json simulation: reconcile the stale stubs the docs pass found 2026-07-26 14:41:37 -07:00
params_lnd_no_contagion.json simulation/lab: retract exp-016's mechanism, three guesses deep 2026-07-26 23:44:36 -07:00
preflight.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
README.md simulation: point README at durable gepa clone and uv 2026-07-24 13:04:18 -07:00
refresh_dashboard.sh simulation/command-center: add paradigm-ceiling section and drift page 2026-07-24 17:31:33 -07:00
run_gepa.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
run_gepa_code.py simulation: run codex searcher agents at xhigh reasoning effort 2026-07-27 00:07:29 -07:00
run_gepa_omni.py simulation: rework the omni runner into the exp-018 adjudication 2026-07-27 00:14:31 -07:00
sweep_validate.py simulation/lab: turn the knob WHY.md said we never turned 2026-07-26 13:55:48 -07:00
warmup_curve.py simulation: add the warmup-curve tool and the scope-vs-clock note 2026-07-26 02:11:12 -07:00

Routing Optimization Harness

This directory holds the GEPA-based optimization harness for lnd's pathfinding. The core idea: lnd's real routing code (or a candidate replacement algorithm) runs against an in-process simulated Lightning Network with hidden liquidity, an evaluator scores the outcome, and a reflective LLM optimizer (GEPA) proposes improved candidates from the failure feedback.

Components

Piece Where What
Simulator routing/sim_*.go In-memory LN with hidden balances; real pathfinding + mission control run unmodified against it
CLI cmd/routesim params JSON + scenario file in, attempt traces + aggregate JSON out
Candidate slot cmd/routesim/candidate_impl.go A complete routing algorithm behind --router=candidate; swapped per candidate via go build -overlay
Corpus gen_scenarios.py train/val/test scenario files: topology + liquidity seed + payment batch
Evaluators evaluate.py, evaluate_code.py score = success rate small saturating penalties for attempts and fee ppm
Runners run_gepa.py, run_gepa_code.py parameter mode and code mode optimization
Reflection LM codex_lm.py GEPA LM protocol via codex exec headless (default gpt-5.6-sol)
Lab notebook lab/ running log of experiments, results, ideas

Quick start

# Build the simulator binary.
go build -o /tmp/routesim ./cmd/routesim

# Generate a scenario corpus.
python3 simulation/gen_scenarios.py --out /tmp/corpus

# Score the lnd defaults on one example.
cd simulation && ROUTESIM_BIN=/tmp/routesim python3 evaluate.py /tmp/corpus/val/example_000.json

# Compare lnd stack vs the candidate router on a scenario file.
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=lnd    --traces=false
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=candidate --traces=false

# Full optimization runs. gepa must be installed from git main — a
# durable clone lives at ~/codez/gepa; prefer uv for the env:
#   uv venv /tmp/gepa-venv && uv pip install -p /tmp/gepa-venv \
#       "~/codez/gepa[full]"
# Also needs the codex CLI authenticated and OPENAI_API_KEY set.
ROUTESIM_BIN=/tmp/routesim python3 run_gepa.py --corpus /tmp/corpus --name run1 --max-evals 400
ROUTESIM_BIN=/tmp/routesim python3 run_gepa_code.py --corpus /tmp/corpus --name code1

The two optimization modes

  1. Parameter mode (run_gepa.py) — candidate = JSON of the existing heuristic's knobs (estimator choice, apriori/bimodal params, attempt cost, min probability). Validates the loop and tunes the current paradigm.
  2. Code mode (run_gepa_code.py) — candidate = the full Go source of candidate_impl.go, an entire routing algorithm implementing the routing.SimRouter interface. This is the paradigm-free path: the candidate sees only gossip, its own balances, and per-attempt feedback. Compile errors are returned to the proposer as feedback.

Anti-reward-hacking measures

  • Candidate routers receive a SimNetworkView wrapper, not the concrete graph, so hidden balances and liquidity mutation are unreachable.
  • evaluate_code.py rejects candidates using unsafe, reflect, os/exec, network packages, etc.
  • Selection happens on a val split; a sealed test split is only used for final reporting.
  • The source's own channels are rebalanced 50/50 before each batch so scores measure routing skill, not sender funding luck.

Command center

command-center/ holds a static dashboard site (serve with python3 -m http.server from that directory).