mirror of
https://github.com/lightningnetwork/lnd.git
synced 2026-08-20 13:27:27 +02:00
5a4e256998
In this commit, we write up exp-010b on the findings page as a new section 07, "The honest arena: atomic commitment, and the fifth challenge", with the process and timeline sections renumbered to 08 and 09 behind it. The section tells the arena story first: shards that hold liquidity until the whole payment settles, siblings contending for what is held, and background traffic drifting on every attempt boundary, so a long reactive ladder finally pays for the churn it sits through. Flag-off byte-identity means none of the earlier results on the page moved. The headline is the baseline, which reordered the field before evolution ran at all: lnd falls from second place to last, spending 105 attempts per payment where it spent 23 with instant settlement, and exp-010's persistent-plan router pulls statistically even with mx_c3 on both atomic tiers without ever having seen an atomic shard. Two tables carry the numbers, the seven-router baseline and the six-tier paired sweep with sign-test deltas against the champion. Then the verdicts. mx_c3 survives its fifth direct challenge on an arena built expressly against its evidence ladder, but the shape of the frontier changed: the codex arm's hybrid of cross-payment memory and reservation-ledger planning is the first challenger in the program with no collapse tier, and it routes mainnet payments in 1.6 attempts, the lowest figure we have measured. The Opus arm lost outright, its drift-bred bound relaxation burning 57 attempts per payment, which flips the exp-010 proposer A/B and adds the clause that proposer strength interacts with environment variance. We also close the forward pointers. The exp-010 sidenote now says how its designed follow-up turned out, the drift page's "one drift intensity" caveat gets the attempt-boundary answer, and the index byline and live-run panel move to exp-010b closed with exp-012 next. |
||
|---|---|---|
| .. | ||
| champions | ||
| command-center | ||
| lab | ||
| .gitignore | ||
| claude_lm.py | ||
| codex_lm.py | ||
| evaluate.py | ||
| evaluate_code.py | ||
| export_run.py | ||
| gen_mainnet_scenarios.py | ||
| gen_scenarios.py | ||
| params_lnd_bimodal.json | ||
| preflight.py | ||
| README.md | ||
| refresh_dashboard.sh | ||
| run_gepa.py | ||
| run_gepa_code.py | ||
| run_gepa_omni.py | ||
| sweep_validate.py | ||
Routing Optimization Harness
This directory holds the GEPA-based optimization harness for lnd's pathfinding. The core idea: lnd's real routing code (or a candidate replacement algorithm) runs against an in-process simulated Lightning Network with hidden liquidity, an evaluator scores the outcome, and a reflective LLM optimizer (GEPA) proposes improved candidates from the failure feedback.
Components
| Piece | Where | What |
|---|---|---|
| Simulator | routing/sim_*.go |
In-memory LN with hidden balances; real pathfinding + mission control run unmodified against it |
| CLI | cmd/routesim |
params JSON + scenario file in, attempt traces + aggregate JSON out |
| Candidate slot | cmd/routesim/candidate_impl.go |
A complete routing algorithm behind --router=candidate; swapped per candidate via go build -overlay |
| Corpus | gen_scenarios.py |
train/val/test scenario files: topology + liquidity seed + payment batch |
| Evaluators | evaluate.py, evaluate_code.py |
score = success rate − small saturating penalties for attempts and fee ppm |
| Runners | run_gepa.py, run_gepa_code.py |
parameter mode and code mode optimization |
| Reflection LM | codex_lm.py |
GEPA LM protocol via codex exec headless (default gpt-5.6-sol) |
| Lab notebook | lab/ |
running log of experiments, results, ideas |
Quick start
# Build the simulator binary.
go build -o /tmp/routesim ./cmd/routesim
# Generate a scenario corpus.
python3 simulation/gen_scenarios.py --out /tmp/corpus
# Score the lnd defaults on one example.
cd simulation && ROUTESIM_BIN=/tmp/routesim python3 evaluate.py /tmp/corpus/val/example_000.json
# Compare lnd stack vs the candidate router on a scenario file.
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=lnd --traces=false
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=candidate --traces=false
# Full optimization runs. gepa must be installed from git main — a
# durable clone lives at ~/codez/gepa; prefer uv for the env:
# uv venv /tmp/gepa-venv && uv pip install -p /tmp/gepa-venv \
# "~/codez/gepa[full]"
# Also needs the codex CLI authenticated and OPENAI_API_KEY set.
ROUTESIM_BIN=/tmp/routesim python3 run_gepa.py --corpus /tmp/corpus --name run1 --max-evals 400
ROUTESIM_BIN=/tmp/routesim python3 run_gepa_code.py --corpus /tmp/corpus --name code1
The two optimization modes
- Parameter mode (
run_gepa.py) — candidate = JSON of the existing heuristic's knobs (estimator choice, apriori/bimodal params, attempt cost, min probability). Validates the loop and tunes the current paradigm. - Code mode (
run_gepa_code.py) — candidate = the full Go source ofcandidate_impl.go, an entire routing algorithm implementing therouting.SimRouterinterface. This is the paradigm-free path: the candidate sees only gossip, its own balances, and per-attempt feedback. Compile errors are returned to the proposer as feedback.
Anti-reward-hacking measures
- Candidate routers receive a
SimNetworkViewwrapper, not the concrete graph, so hidden balances and liquidity mutation are unreachable. evaluate_code.pyrejects candidates usingunsafe,reflect,os/exec, network packages, etc.- Selection happens on a val split; a sealed test split is only used for final reporting.
- The source's own channels are rebalanced 50/50 before each batch so scores measure routing skill, not sender funding luck.
Command center
command-center/ holds a static dashboard site (serve with
python3 -m http.server from that directory).