lnd/simulation
Olaoluwa Osuntokun 820d06d016 simulation: searchers default to high effort, 900s reflections
In this commit, we retune the codex searcher defaults with exp-018's
measurements in hand: the gepa arm at xhigh lost four of thirteen
iterations to the 600s reflection timeout and took nine hours for 150
evals. The evolutionary loop supplies the search, so iteration
throughput beats per-proposal depth; the default effort drops back to
high with a 900s timeout for headroom on large seeds, and xhigh stays
one flag away via codex:<model>:xhigh for runs where a deep single
proposal is the point.
2026-07-27 10:53:04 -07:00
..
champions simulation: reconcile the stale stubs the docs pass found 2026-07-26 14:41:37 -07:00
command-center simulation/lab: exp-019b bounds the anomaly and kills its story 2026-07-27 01:48:10 -07:00
lab simulation/lab: exp-019b bounds the anomaly and kills its story 2026-07-27 01:48:10 -07:00
.gitignore simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
claude_lm.py simulation: isolate claude -p reflections from user-level hooks 2026-07-25 13:01:36 -07:00
codex_lm.py simulation: searchers default to high effort, 900s reflections 2026-07-27 10:53:04 -07:00
evaluate.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
evaluate_code.py simulation: recalibrate the abandonment hint 2026-07-27 00:21:54 -07:00
export_run.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
gen_family_corpora.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_mainnet_scenarios.py simulation: adopt advisor corrections to measurement and validation 2026-07-25 02:58:52 -07:00
gen_mainnet_variants.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_scenarios.py routing+routesim: degrade the failure attribution channel 2026-07-27 00:54:37 -07:00
gen_served_weights.py simulation/lab: exp-016, free knowledge helps the champions and hurts lnd 2026-07-26 23:06:50 -07:00
gen_warmup_scenarios.py routing+routesim: separate what a warmup teaches from what it spends 2026-07-26 02:38:57 -07:00
params_lnd_bimodal.json simulation: reconcile the stale stubs the docs pass found 2026-07-26 14:41:37 -07:00
params_lnd_no_contagion.json simulation/lab: retract exp-016's mechanism, three guesses deep 2026-07-26 23:44:36 -07:00
preflight.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
README.md simulation: point README at durable gepa clone and uv 2026-07-24 13:04:18 -07:00
refresh_dashboard.sh simulation/command-center: add paradigm-ceiling section and drift page 2026-07-24 17:31:33 -07:00
run_gepa.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
run_gepa_code.py simulation: searchers default to high effort, 900s reflections 2026-07-27 10:53:04 -07:00
run_gepa_omni.py simulation: rework the omni runner into the exp-018 adjudication 2026-07-27 00:14:31 -07:00
sweep_validate.py simulation/lab: turn the knob WHY.md said we never turned 2026-07-26 13:55:48 -07:00
warmup_curve.py simulation: add the warmup-curve tool and the scope-vs-clock note 2026-07-26 02:11:12 -07:00

Routing Optimization Harness

This directory holds the GEPA-based optimization harness for lnd's pathfinding. The core idea: lnd's real routing code (or a candidate replacement algorithm) runs against an in-process simulated Lightning Network with hidden liquidity, an evaluator scores the outcome, and a reflective LLM optimizer (GEPA) proposes improved candidates from the failure feedback.

Components

Piece Where What
Simulator routing/sim_*.go In-memory LN with hidden balances; real pathfinding + mission control run unmodified against it
CLI cmd/routesim params JSON + scenario file in, attempt traces + aggregate JSON out
Candidate slot cmd/routesim/candidate_impl.go A complete routing algorithm behind --router=candidate; swapped per candidate via go build -overlay
Corpus gen_scenarios.py train/val/test scenario files: topology + liquidity seed + payment batch
Evaluators evaluate.py, evaluate_code.py score = success rate small saturating penalties for attempts and fee ppm
Runners run_gepa.py, run_gepa_code.py parameter mode and code mode optimization
Reflection LM codex_lm.py GEPA LM protocol via codex exec headless (default gpt-5.6-sol)
Lab notebook lab/ running log of experiments, results, ideas

Quick start

# Build the simulator binary.
go build -o /tmp/routesim ./cmd/routesim

# Generate a scenario corpus.
python3 simulation/gen_scenarios.py --out /tmp/corpus

# Score the lnd defaults on one example.
cd simulation && ROUTESIM_BIN=/tmp/routesim python3 evaluate.py /tmp/corpus/val/example_000.json

# Compare lnd stack vs the candidate router on a scenario file.
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=lnd    --traces=false
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=candidate --traces=false

# Full optimization runs. gepa must be installed from git main — a
# durable clone lives at ~/codez/gepa; prefer uv for the env:
#   uv venv /tmp/gepa-venv && uv pip install -p /tmp/gepa-venv \
#       "~/codez/gepa[full]"
# Also needs the codex CLI authenticated and OPENAI_API_KEY set.
ROUTESIM_BIN=/tmp/routesim python3 run_gepa.py --corpus /tmp/corpus --name run1 --max-evals 400
ROUTESIM_BIN=/tmp/routesim python3 run_gepa_code.py --corpus /tmp/corpus --name code1

The two optimization modes

  1. Parameter mode (run_gepa.py) — candidate = JSON of the existing heuristic's knobs (estimator choice, apriori/bimodal params, attempt cost, min probability). Validates the loop and tunes the current paradigm.
  2. Code mode (run_gepa_code.py) — candidate = the full Go source of candidate_impl.go, an entire routing algorithm implementing the routing.SimRouter interface. This is the paradigm-free path: the candidate sees only gossip, its own balances, and per-attempt feedback. Compile errors are returned to the proposer as feedback.

Anti-reward-hacking measures

  • Candidate routers receive a SimNetworkView wrapper, not the concrete graph, so hidden balances and liquidity mutation are unreachable.
  • evaluate_code.py rejects candidates using unsafe, reflect, os/exec, network packages, etc.
  • Selection happens on a val split; a sealed test split is only used for final reporting.
  • The source's own channels are rebalanced 50/50 before each batch so scores measure routing skill, not sender funding luck.

Command center

command-center/ holds a static dashboard site (serve with python3 -m http.server from that directory).