lnd/simulation
Olaoluwa Osuntokun 39103864c7 simulation/lab: exp-032, margins hold and magnitudes humble
In this commit, we close the simulator program's last experiment.
The release candidate's lead over stock lnd re-pins CI-solid on six
of seven fresh-file tiers at n=30, with the exp-019 collapse
reproducing on unseen files. The exp-030 point estimates do not
re-pin — bug and fix are both indistinguishable from zero on thirty
fresh mix files — so the record now says what survives: the
mechanism and its ground-truth counters, never the deltas. The
degraded-mainnet deficit halves and loses significance while
mx_c3's exact success invariance holds, and the objective's
abandonment subsidy logs its third sighting. The correction is
written into the exp-030 writeup; the PR draft update follows on
the branch.
2026-07-31 00:54:04 -07:00
..
champions simulation/champions: document the specialist roster 2026-07-29 16:38:11 -07:00
command-center simulation/command-center: v66, the boundary and the balance sheet 2026-07-31 00:14:16 -07:00
lab simulation/lab: exp-032, margins hold and magnitudes humble 2026-07-31 00:54:04 -07:00
.gitignore simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
claude_lm.py simulation: isolate claude -p reflections from user-level hooks 2026-07-25 13:01:36 -07:00
codex_lm.py simulation: searchers default to high effort, 900s reflections 2026-07-27 10:53:04 -07:00
evaluate.py simulation: score an archived run on time instead of attempts 2026-07-28 01:34:53 -07:00
evaluate_code.py simulation: name the fee metric, and the rule that guards its weight 2026-07-28 00:15:43 -07:00
export_run.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
gen_family_corpora.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_mainnet_scenarios.py simulation: adopt advisor corrections to measurement and validation 2026-07-25 02:58:52 -07:00
gen_mainnet_variants.py simulation: add the exp-017 family corpus tooling 2026-07-27 00:07:17 -07:00
gen_scenarios.py simulation: stamp latency onto generated corpora 2026-07-28 01:33:17 -07:00
gen_served_weights.py simulation/lab: exp-016, free knowledge helps the champions and hurts lnd 2026-07-26 23:06:50 -07:00
gen_warmup_scenarios.py routing+routesim: separate what a warmup teaches from what it spends 2026-07-26 02:38:57 -07:00
params_lnd_bimodal.json simulation: reconcile the stale stubs the docs pass found 2026-07-26 14:41:37 -07:00
params_lnd_no_contagion.json simulation/lab: retract exp-016's mechanism, three guesses deep 2026-07-26 23:44:36 -07:00
params_lnd_patch.json routing: the distillation patch, one live fix and one measured negative 2026-07-27 11:48:24 -07:00
preflight.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
README.md simulation: point README at durable gepa clone and uv 2026-07-24 13:04:18 -07:00
refresh_dashboard.sh simulation/command-center: add paradigm-ceiling section and drift page 2026-07-24 17:31:33 -07:00
run_gepa.py simulation: add GEPA optimization harness 2026-07-24 13:01:06 -07:00
run_gepa_code.py simulation: tell candidates the inbound fee type, and name the ban 2026-07-28 18:03:06 -07:00
run_gepa_omni.py simulation: rework the omni runner into the exp-018 adjudication 2026-07-27 00:14:31 -07:00
sweep_validate.py simulation/lab: turn the knob WHY.md said we never turned 2026-07-26 13:55:48 -07:00
warmup_curve.py simulation: add the warmup-curve tool and the scope-vs-clock note 2026-07-26 02:11:12 -07:00

Routing Optimization Harness

This directory holds the GEPA-based optimization harness for lnd's pathfinding. The core idea: lnd's real routing code (or a candidate replacement algorithm) runs against an in-process simulated Lightning Network with hidden liquidity, an evaluator scores the outcome, and a reflective LLM optimizer (GEPA) proposes improved candidates from the failure feedback.

Components

Piece Where What
Simulator routing/sim_*.go In-memory LN with hidden balances; real pathfinding + mission control run unmodified against it
CLI cmd/routesim params JSON + scenario file in, attempt traces + aggregate JSON out
Candidate slot cmd/routesim/candidate_impl.go A complete routing algorithm behind --router=candidate; swapped per candidate via go build -overlay
Corpus gen_scenarios.py train/val/test scenario files: topology + liquidity seed + payment batch
Evaluators evaluate.py, evaluate_code.py score = success rate small saturating penalties for attempts and fee ppm
Runners run_gepa.py, run_gepa_code.py parameter mode and code mode optimization
Reflection LM codex_lm.py GEPA LM protocol via codex exec headless (default gpt-5.6-sol)
Lab notebook lab/ running log of experiments, results, ideas

Quick start

# Build the simulator binary.
go build -o /tmp/routesim ./cmd/routesim

# Generate a scenario corpus.
python3 simulation/gen_scenarios.py --out /tmp/corpus

# Score the lnd defaults on one example.
cd simulation && ROUTESIM_BIN=/tmp/routesim python3 evaluate.py /tmp/corpus/val/example_000.json

# Compare lnd stack vs the candidate router on a scenario file.
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=lnd    --traces=false
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=candidate --traces=false

# Full optimization runs. gepa must be installed from git main — a
# durable clone lives at ~/codez/gepa; prefer uv for the env:
#   uv venv /tmp/gepa-venv && uv pip install -p /tmp/gepa-venv \
#       "~/codez/gepa[full]"
# Also needs the codex CLI authenticated and OPENAI_API_KEY set.
ROUTESIM_BIN=/tmp/routesim python3 run_gepa.py --corpus /tmp/corpus --name run1 --max-evals 400
ROUTESIM_BIN=/tmp/routesim python3 run_gepa_code.py --corpus /tmp/corpus --name code1

The two optimization modes

  1. Parameter mode (run_gepa.py) — candidate = JSON of the existing heuristic's knobs (estimator choice, apriori/bimodal params, attempt cost, min probability). Validates the loop and tunes the current paradigm.
  2. Code mode (run_gepa_code.py) — candidate = the full Go source of candidate_impl.go, an entire routing algorithm implementing the routing.SimRouter interface. This is the paradigm-free path: the candidate sees only gossip, its own balances, and per-attempt feedback. Compile errors are returned to the proposer as feedback.

Anti-reward-hacking measures

  • Candidate routers receive a SimNetworkView wrapper, not the concrete graph, so hidden balances and liquidity mutation are unreachable.
  • evaluate_code.py rejects candidates using unsafe, reflect, os/exec, network packages, etc.
  • Selection happens on a val split; a sealed test split is only used for final reporting.
  • The source's own channels are rebalanced 50/50 before each batch so scores measure routing skill, not sender funding luck.

Command center

command-center/ holds a static dashboard site (serve with python3 -m http.server from that directory).