mirror of
https://github.com/lightningnetwork/lnd.git
synced 2026-08-20 13:27:27 +02:00
11f4ccc65d
In this commit, we add the exp-010 environment: a corridors topology that makes splitting mandatory and makes the right split unequal. K parallel corridors run from a single source to a single target, each terminating in one tier channel into the target, and the target has no other channels -- so the fattest tier is a hard structural ceiling on any single shard and the sum of tiers a hard ceiling on the whole payment, independent of liquidity luck. The tier ladder puts one uniquely fat corridor above repeating rungs each at most half its size, which is what punishes blind halving: half of an above-tier payment fits only the fat corridor, so a divide-and-conquer splitter must keep halving while a deliberate splitter sizes shards to tiers. Interior hops are 128x fatter than the tiers so the tier stays the binding constraint, a low-capacity filler cloud tempts fee-greedy routers without being able to carry a real shard, and corridor fees rise with tier so cheapest-first search pulls toward corridors that cannot carry the payment. The corpus generator gains --split: corridors networks under bimodal liquidity with no drift, two cheap probe payments that seed corridor knowledge, then one ambitious payment above the fattest tier, sized against a measured usable-capacity budget so files discriminate rather than saturate. End-to-end the corpus already separates strategies sharply: lnd's production halving completes 75% of the ambitious payments while the seed's naive halving completes 38%, a role reversal against every other corpus, and a forced max_parts=1 control fails 40 of 40 files, confirming the structural guarantee. Determinism, structure, and behavioral tests cover the generator. |
||
|---|---|---|
| .. | ||
| champions | ||
| command-center | ||
| lab | ||
| .gitignore | ||
| codex_lm.py | ||
| evaluate.py | ||
| evaluate_code.py | ||
| export_run.py | ||
| gen_scenarios.py | ||
| preflight.py | ||
| README.md | ||
| refresh_dashboard.sh | ||
| run_gepa.py | ||
| run_gepa_code.py | ||
| run_gepa_omni.py | ||
Routing Optimization Harness
This directory holds the GEPA-based optimization harness for lnd's pathfinding. The core idea: lnd's real routing code (or a candidate replacement algorithm) runs against an in-process simulated Lightning Network with hidden liquidity, an evaluator scores the outcome, and a reflective LLM optimizer (GEPA) proposes improved candidates from the failure feedback.
Components
| Piece | Where | What |
|---|---|---|
| Simulator | routing/sim_*.go |
In-memory LN with hidden balances; real pathfinding + mission control run unmodified against it |
| CLI | cmd/routesim |
params JSON + scenario file in, attempt traces + aggregate JSON out |
| Candidate slot | cmd/routesim/candidate_impl.go |
A complete routing algorithm behind --router=candidate; swapped per candidate via go build -overlay |
| Corpus | gen_scenarios.py |
train/val/test scenario files: topology + liquidity seed + payment batch |
| Evaluators | evaluate.py, evaluate_code.py |
score = success rate − small saturating penalties for attempts and fee ppm |
| Runners | run_gepa.py, run_gepa_code.py |
parameter mode and code mode optimization |
| Reflection LM | codex_lm.py |
GEPA LM protocol via codex exec headless (default gpt-5.6-sol) |
| Lab notebook | lab/ |
running log of experiments, results, ideas |
Quick start
# Build the simulator binary.
go build -o /tmp/routesim ./cmd/routesim
# Generate a scenario corpus.
python3 simulation/gen_scenarios.py --out /tmp/corpus
# Score the lnd defaults on one example.
cd simulation && ROUTESIM_BIN=/tmp/routesim python3 evaluate.py /tmp/corpus/val/example_000.json
# Compare lnd stack vs the candidate router on a scenario file.
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=lnd --traces=false
/tmp/routesim --scenarios /tmp/corpus/val/example_000.json --router=candidate --traces=false
# Full optimization runs. gepa must be installed from git main — a
# durable clone lives at ~/codez/gepa; prefer uv for the env:
# uv venv /tmp/gepa-venv && uv pip install -p /tmp/gepa-venv \
# "~/codez/gepa[full]"
# Also needs the codex CLI authenticated and OPENAI_API_KEY set.
ROUTESIM_BIN=/tmp/routesim python3 run_gepa.py --corpus /tmp/corpus --name run1 --max-evals 400
ROUTESIM_BIN=/tmp/routesim python3 run_gepa_code.py --corpus /tmp/corpus --name code1
The two optimization modes
- Parameter mode (
run_gepa.py) — candidate = JSON of the existing heuristic's knobs (estimator choice, apriori/bimodal params, attempt cost, min probability). Validates the loop and tunes the current paradigm. - Code mode (
run_gepa_code.py) — candidate = the full Go source ofcandidate_impl.go, an entire routing algorithm implementing therouting.SimRouterinterface. This is the paradigm-free path: the candidate sees only gossip, its own balances, and per-attempt feedback. Compile errors are returned to the proposer as feedback.
Anti-reward-hacking measures
- Candidate routers receive a
SimNetworkViewwrapper, not the concrete graph, so hidden balances and liquidity mutation are unreachable. evaluate_code.pyrejects candidates usingunsafe,reflect,os/exec, network packages, etc.- Selection happens on a val split; a sealed test split is only used for final reporting.
- The source's own channels are rebalanced 50/50 before each batch so scores measure routing skill, not sender funding luck.
Command center
command-center/ holds a static dashboard site (serve with
python3 -m http.server from that directory).