docs: restructure — 9 files→3, compelling entry point

- README: rewrite as compelling 60-line entry (what, why, setup, status)
- ARCHITECTURE: absorb KNOWN-ISSUES as appendix
- Delete 5 redundant docs (BACKEND-AUDIT, COMPARISON-MATRIX,
  GL-BACKEND-REPORT, UPSTREAM-MERGE, mainnet-test-plan)
- Keep: RUNBOOK.md (operations)
This commit is contained in:
welliv 2026-08-08 22:55:08 +00:00
parent c7bdeebe9e
commit b8f34d4728
8 changed files with 130 additions and 597 deletions

View file

@ -1,97 +0,0 @@
# Backend audit: Greenlight vs the other Alby Hub backends
> **Note (2026-08-08):** The method-level audit below remains accurate, but for
> the current feature set (signer backup, health events, watchdog surfacing)
> see **[ARCHITECTURE.md](../lnclient/greenlight/ARCHITECTURE.md)** and
> **[COMPARISON-MATRIX.md](COMPARISON-MATRIX.md)**.
Audit date: 2026-08-07. Base: upstream getAlby/hub @ 6175489 (LDK/CLN/LND/Phoenixd/Cashu/Bark), fork feat/greenlight-backend (GREENLIGHT). Method-level conformance verified by source read + live tests.
## Interface conformance (35-method LNClient)
Real implementations (stubs/errors excluded; verified by source read, conditional errors counted as implemented):
| Backend | Real | Stubs | Notable stubs |
|---|---|---|---|
| **LND** | 31 | 4 | MakeOffer (BOLT12 unsupported) |
| **CLN** | 30 | 5 | holds ×3 (gated on external hold plugin), ResetRouter, **SendKeysend** (preimage bug fixed via PR #2521, pending merge) |
| **GREENLIGHT** | 30 | 5 | holds ×3, ResetRouter, **SignMessage (stubbed: VLS freeze)** |
| **LDK** | 33+ | ~2 | GetNetworkGraph stub (LDK has no graph query API) |
| **Phoenixd** | ~13 | ~22 | channel ops, keysend, onchain — mostly unavailable (managed node) |
| **Bark** | ~13 | ~22 | no channels by design (VTXO/Ark), no invoices lookup |
| **Cashu** | ~10 | ~25 | ecash wallet: minimal surface |
Greenlight is the **second-most complete node-model backend** (behind LDK/LND), with the widest surface of the *hosted* backends.
## Feature matrix (empirically verified where possible)
| Feature | LDK | CLN | LND | Phoenixd | Cashu | Bark | **GL** |
|---|---|---|---|---|---|---|---|
| Hold invoices | ✅ | ⚠️ plugin-gated | ✅ | ❌ | ❌ | ❌ | ❌ |
| BOLT12 offers | ✅ | ✅ | ❌ stub | ❌ | ❌ | ❌ | ✅ |
| Keysend out | ✅ | ✅ (fix PR #2521) | ✅ | ❌ | ❌ | ❌ | ✅ (fixed) |
| SignMessage | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ (VLS freeze) |
| Channels open/close | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ (VTXO) | ✅ |
| Onchain (addr/withdraw) | ✅ | ✅ | ✅ | ❌ | ❌ | ⚠️ | ✅ |
| Network graph | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ |
| payment_received notif | ❌ (reconcile) | ❌ (reconcile) | ❌ (reconcile) | ❌ | ❌ | ❌ | ✅ **(pump)** |
| NIP-47 advertised | 14 | 11 | 14 | partial | partial | partial | 10 |
| GetStorageDir-aware backup | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ |
## Incoming-payment architecture (the key design axis)
| Backend | Mechanism | Restart safety |
|---|---|---|
| LDK | embedded LDK events | channel monitors + reconcile |
| CLN | `SubscribeInvoices` stream | ❌ ephemeral; relies on hub reconcile poll |
| LND | lnrpc SubscribeInvoices | relies on hub reconcile poll |
| **GL** | **WaitAnyInvoice pump + persisted pay_index** (atomic tmp+rename, EXPIRED fast-forward) | ✅ **proven**: kill → resume from index, no missed/dup payments |
| Bark | Ark "movements" | n/a |
Why GL differs: the hosted GL node leaves all six `cln.Node Subscribe*` streams `unimplemented!()` (they panic the gl-plugin), so the fork adapted the poll-based WaitAnyInvoice — and in doing so added persistence the upstream CLN backend lacks. Advertising `payment_received` (unique among backends) disables the hub's reconcile safety net; the pump's index catch-up replaces it. **This is the fork's strongest architectural contribution.**
## Bugs & quirks inventory (ours vs theirs)
| Backend | Issue | Status |
|---|---|---|
| GL | NIP-47 pay_keysend preimage rejection | ✅ **fixed in fork** (9e421612) + **live-verified end-to-end** (2 keysends settled on fresh harness, l1 confirmed, hub recorded outgoing/settled) |
| GL | sign_message freezes node (hsmd wedge) | ✅ **stubbed** (9e421612); Blockstream issue drafted |
| GL | extract_creds.py CWD/hardcoded path | ✅ **fixed** (aa71d61f, go:embed) |
| GL | keysend TLV metadata lost (stream guard never fires) | ⚠️ documented; design debt |
| GL | ConnectPeer address+port=0 concat footgun | ⚠️ minor (UI sends split fields) |
| GL | **mutual close** | ✅ **works** — l1 state history shows CLOSINGD_COMPLETE (2250 sats fee); earlier "stall" was the LDK hub, not GL |
| GL | keysend preimage in tx record is synthetic (hub-generated; CLN derives the real one internally, backend doesn't return it) | ⚠️ accounting nuance — payment settles + real preimage exists on peer side, but hub's recorded preimage ≠ real one |
| CLN | SendKeysend preimage rejection (same bug GL had) | ❌ **still live upstream** — PR candidate |
| CLN | hold methods never advertised even with plugin | ⚠️ minor |
| LDK | `testnet` maps to `ldk_node.NetworkSignet` | ❌ live upstream quirk |
| LDK | withdraw 500 "ran out of attempts to fetch broadcasted transaction" (funds still move) | ❌ live upstream |
| LDK | mutual-close stall vs CLN peer on regtest (force works) | ⚠️ interop |
| LND | MakeOffer stub (no BOLT12) | by design |
| Bark | sqlite 644 perms + VTXO recovery-mailbox message | ⚠️ fresh-wallet noise |
## Verdict
1. **Conformance:** GL is in the top tier — 30/35 methods real, everything a node backend needs (channels, onchain, offers, peers, graph, log). Nothing is stubbed that shouldn't be (holds are the only feature gap; sign_message is stubbed for a documented safety reason).
2. **Unique strengths:** restart-safe incoming pump (only backend with persisted index + payment_received notifications), signer supervision, GL-aware backup/UI.
3. **Cleanest of the CLN family:** GL fixed the keysend bug upstream CLN still has, and the deployment bug is now embed-based.
4. **Actionable gaps:** holds (blocked upstream), sign_message (blocked upstream), keysend TLV capture (fork-side design work), and two upstream PR candidates: CLN keysend preimage bug + LDK testnet→signet mapping.
## Update 2026-08-08 (Phase A hardening)
- **Node health watchdog** (health.go): every 30s the GL backend probes the node (Getinfo, 10s
deadline) and watches the invoice pump's outstanding WaitAnyInvoice call (>90s = hsmd-wedge
symptom). After 3 consecutive failures it logs ERROR + publishes `nwc_gl_node_health`; the
verdict is cached and served by GetNodeStatus, so /api/node/status keeps answering (21ms,
measured live) even when the node is frozen. Verified live: a genuine #739 freeze reproduced on
the regtest harness (signmessage → every RPC hung) was detected; the hub booted degraded instead
of aborting (user never locked out of their wallet UI).
- **Boot behavior change**: a frozen/unreachable node no longer aborts hub startup — the backend
starts degraded and the watchdog reports the state. (Previously the 15s connectivity probe
failing = LNClient never started = no UI, no backups, no apps — the worst time to be locked out.)
- **Keysend TLVs (correction)**: incoming keysend custom records ARE captured — streamIncoming
maps `offchain.ExtraTLVs` into the transaction Metadata (`tlv_records`) and they surface to
NIP-47 clients via the freeform metadata blob. Earlier "TLV loss" note was stale.
- **Hold invoices**: confirmed unsupported at the stack level (no hold hooks in gl-client); hosted
GL nodes run no plugins. Upstream-blocked; do not ship a fake hold path.
- **ResetRouter**: stub retained (graph lives on the hosted node, no reset RPC); the backup path
tolerates it (warns and continues, backup.go:63).

View file

@ -1,29 +0,0 @@
# Three-way comparison matrix: Greenlight standalone vs Hub standalone vs our GL backend
Status: **populated with empirical evidence** (2026-08-07, regtest harness + testnet + vanilla hub deep-dive). Each cell = observed behavior.
Scope: default flows on regtest/testnet with recorded evidence.
| Flow | Upstream GL standalone (glcli) | Upstream Hub standalone (LDK) | Our GL backend | Verdict |
|---|---|---|---|---|
| Register node (dev cert) | `glcli scheduler register` w/ GL_NOBODY_CRT/KEY → credentials.gfs | n/a (embedded node) | `EnsureProvisioned` runs glcli register/recover inside hub setup (UI wizard) | ✅ Ours reuses proven upstream path, zero-tool UX |
| Recovery from 12-word seed | `scheduler recover` → same node_id | n/a | Setup "recover" path + live re-register test: same pubkey, byte-identical seed derivation | ✅ Matches upstream semantics |
| Node getinfo / identity | scheduler-issued node domain | LDK pubkey 02febb… | GL node pubkey via gRPC (02ce79da testnet / 026f61d7 regtest) | ✅ |
| Receive invoice (bolt11) | invoice via cln RPC | LDK event → settled | pump (WaitAnyInvoice, persisted index) → settled; 250k msat preimage captured | ✅ pump is restart-safe (proven kill→resume) |
| Send payment | pay via Xpay | LDK pathfinder (needs outbound channel) | Xpay → settled, preimage returned, peer confirms | ✅ |
| Keysend | keysend RPC | LDK custom-preimage keysend works | **incoming** via streamIncoming (TLVs captured in transaction Metadata, surfaced to NIP-47); **outgoing NIP-47 fixed** (records actual preimage/hash from the KeySend response) | ✅ after fix; verified live |
| Balance / on-chain addr | listfunds | onchain+channel | GetBalances + NewAddr via cln-grpc; withdraw proven (0.9996 BTC swept) | ✅ |
| Channel open/close | fundchannel/close | open both directions, force-close proven | open (fundchannel), close (mutual stalled vs CLN peer, force works), 2 channels live | ✅ with noted close quirk |
| Signer lifecycle | `glcli signer run` (VLS) | n/a | supervised (15s ticker, respawn proven, SIGTERM→KILL + native `--backup-path`); publishes `nwc_gl_signer_health` events | ✅ GL-native (backup snapshots + health events) |
| Node health | — | — | Watchdog (30s Getinfo + pump-stall detection, `nwc_gl_node_health` events); degraded boot (hub survives frozen node) | 🔵 No other backend has this |
| Backup / restore | `glcli signer run --backup-path` + `scheduler recover` | `.bkp` 40KB (LDK state) | `.bkp` includes `SignerDataDir` (seed + creds + `backup.json`); `backup.json` → CLN `recoverchannel` escape hatch | ✅ GL-native backup included; CLN escape hatch documented |
| NWC app pairing | n/a | full NIP-47 matrix proven | full matrix on GL (balance/invoice/pay/keysend/budget); QUOTA_EXCEEDED proven | ✅ |
| Isolated sub-wallets | n/a | hub feature | same hub code path | ✅ (untested, shared) |
| Lightning address | n/a | hub+Alby account | requires Alby account (skipped in tests) | ⏳ untested |
| UI setup wizard | glcli only | hub wizard (LDK) | GL-aware wizard: Get Started (GREENLIGHT), 12-word entry, GL Backup screen w/ recovery phrase modal, "Leave Greenlight" | ✅ tested in browser |
## Notes
- All cells observed live (command/API response/log line recorded in session); nothing assumed.
- Our GL backend fixes two upstream CLN-family bugs: NIP-47 pay_keysend (preimage rejection) and deployment paths (embedded extract_creds.py).
- Known deliberate gaps: hold invoices (unsupported), sign_message (stubbed — VLS signer freezes node; issue drafted for Blockstream).

View file

@ -1,70 +0,0 @@
# Greenlight backend for Alby Hub: what and why
> **Note (2026-08-08):** This is the initial report from the first build.
> For the current architecture, design rationale, and integration surface,
> see **[ARCHITECTURE.md](../lnclient/greenlight/ARCHITECTURE.md)**.
> This document is kept for historical reference; the ARCHITECTURE.md and
> RUNBOOK.md are the canonical sources.
Status snapshot: 2026-08-07, branch `feat/greenlight-backend`, commits `535e66e` + `19ec7cc` on upstream base `bf9c346`.
## Goal
Make Greenlight a first-class LN backend inside Alby Hub. Cloud node (Blockstream CLN) with self-custody keys on the device. Custody split:
- Hub owns: control plane, NWC, encrypted 12-word mnemonic, supervised signer process, payment pump
- Blockstream owns: lightningd and the channel DB
Why this shape: LDK is fully local and heavy, Bark/Ark has no channels. Greenlight gives an always-on node with real channels and user-held keys. Hub supervises the signer the same way Routstrd supervises its daemons.
## Research findings (source-verified)
- Node seed = 12-word BIP-39 mnemonic, derived as `to_seed("")[0:32]` (gl-cli `util.rs`). One-way derivation, 24-word phrases rejected
- Recovery: seed alone restores node identity, device creds, on-chain funds. Channels additionally need the signer's periodic backup file
- Signer is a local process; without it money RPCs hang. Supervision pattern mirrors `RoutstrdService` (15s health ticker, respawn)
- GL networks: bitcoin, testnet, regtest. Signet unsupported (live TLS 223)
## Build
Backend (`lnclient/greenlight/`):
- `greenlight.go`: full 36-method LNClient over GL gRPC (invoices, pays, keysend, balance, channels, waitanyinvoice, streams)
- `provision.go`: mnemonic to seed, register/recover via Nobody dev cert, schedule, extract mTLS PEMs + node URI
- `service/gl_signer.go`: supervised `glcli signer run` (health loop, respawn, clean kill)
- `service/start.go`: product startup path, network mapping fix (testnet passes through)
- Wired at the 4 boot-contract points: config/models.go, config/config.go, service/start.go, constructor
Frontend:
- `GreenlightForm.tsx` (12-word setup), `SetupRecover.tsx`, GL-aware `Backup.tsx`, `MigrateNode.tsx`
- Registered with `hasMnemonic: true`, `hasChannelManagement: true`, `hasNodeBackup: false`
Delta: 40 files, +5243/-131.
## Testing ladder
1. Mock suite (23 tests, in-process bufconn gRPC)
2. Local gl-testing harness (regtest): real lightningd + gl-plugin + VLS signer proxy, v25.05gl1
3. Two-node in-harness E2E: channel opened, 2x50k received + 1x10k sent settled
4. TestLiveE2E gated behind GREENLIGHT_LIVE=1
## Live testnet results
- Registered real Blockstream testnet nodes with dev Nobody certs, mTLS gRPC connected
- Fixed real bug: start.go forced non-mainnet to signet, signer would be read-only on testnet
- Redeployed from committed source, deployed binary == ./hub byte-for-byte
- Minted lntb invoices live (proves signer attached)
- Deterministic restore: two scheduler recover runs from one phrase reproduce the same node pubkey
## Verified gaps (honest)
- Nothing pushed upstream; no PR (this private repo is the housing)
- Live send on testnet blocked: fresh node has zero on-chain, all public testnet faucets down
- Mainnet channel + fund path not exercised
## Three-way comparison (next phase)
See `COMPARISON-MATRIX.md`. Deploy upstream Greenlight standalone, upstream Hub standalone, run default flows on testnet, compare against this backend.
## Excluded from repo (operator secrets stay local)
- `/root/gl-tools/certs` (GL_NOBODY_CRT/KEY)
- `.data`, `.env`, compiled binaries

View file

@ -1,76 +0,0 @@
# Greenlight backend — known issues (honest gaps)
Status of each issue: **blocked** (upstream dependency), **mitigated**
(workaround shipped), or **fixed** (resolved + verified).
## 1. signmessage freezes the production signer — BLOCKED (upstream #739)
`signmessage` sends a signing request the production VLS signer cannot
answer; hsmd's queue wedges and the whole node freezes until Blockstream
restarts it. Reproduced twice (hosted testnet node A; regtest harness,
2026-08-08). Filed as [Blockstream/greenlight#739](https://github.com/Blockstream/greenlight/issues/739)
— unacknowledged.
- Mitigation: the backend stubs `SignMessage` (returns a clear error,
never forwards). The health watchdog (health.go) detects a freeze and the
hub boots degraded instead of locking the user out. RUNBOOK.md has the
recovery path.
- Unblocks when: Blockstream fixes the signer (or the issue is formally
acknowledged + a workaround documented).
## 2. Hold invoices — BLOCKED (stack-level)
Greenlight hosted nodes run no plugins and gl-client exposes no hold hooks
(verified by grep, 2026-08-08). `MakeHoldInvoice`/`Settle`/`Cancel` return a
clear error. Only feature gap vs LDK/CLN/LND. Do not ship a fake hold path.
## 3. Incoming keysend TLV records — FIXED (correction)
Earlier audits noted "TLVs dropped". Correction: `streamIncoming` maps the
incoming keysend `ExtraTLVs` into the transaction `Metadata` (`tlv_records`)
and they surface to NIP-47 clients via the freeform metadata blob. Verified
in source (streamincoming.go:87-96) + models.go comment.
## 4. Outgoing keysend preimage/hash recording — FIXED
CLN-family keysend derives its own preimage server-side; the caller's
preimage cannot be honored (no gRPC field). The backend now reports the
actual preimage + payment hash from the KeySend response and the
transactions service records those — the stored hash is the one that
actually settled (verified live on CLN + GL, 2026-08-08; upstream PR #2521).
## 5. ResetRouter — MITIGATED
The hosted node owns the network graph; there is no reset RPC. The stub
returns a clear error; the backup path tolerates it (warns + continues,
api/backup.go:63). Only the explicit reset-router API endpoint surfaces it.
## 6. Node hosted elsewhere — INHERENT (design)
The node's availability depends on Blockstream infrastructure (the flip
side of the custody split). The watchdog + runbook cover detection and
recovery; a frozen node cannot be restarted by the user.
## 7. channel-state notifications — MITIGATED
The GL node server leaves all six `cln.Node Subscribe*` stream RPCs
unimplemented (they panic the gl-plugin process). `nwc_channel_ready/closed`
notifications are not emitted; incoming payments are covered by the
restart-safe pump instead.
## 8. Signer seed at rest — PLAINTEXT (posture note, mainnet audit 2026-08-08)
The hub supervisor spawns `glcli signer run` directly, so the seed
(`hsm_secret`) sits plaintext in the 0700 data dir while the hub is
unlocked — the same posture as the hub's own LDK seed. An at-rest
encryption wrapper exists (`lnclient/greenlight/signer/encrypt-seed.py` +
`greenlight-signer-run.sh`, AES-256-GCM, for systemd deployments) but is
not wired into the supervisor. Wiring it is a follow-up; for a single-user
device deployment the 0700/0600 file perms match the rest of the wallet.
Audit-verified clean: dataDir 0700, hsm_secret/signer.log/signer.pid 0600,
device PEMs 0600, seed write-once (a different mnemonic can never desync
the signer from the node), signer liveness tracks the live process.
See [ARCHITECTURE.md](../lnclient/greenlight/ARCHITECTURE.md) for the
full design rationale (why we integrate through `glcli`, how GL primitives
map to the Go layer, and what operational additions we provide).

View file

@ -1,158 +0,0 @@
# Upstream sync & push workflow (verified 2026-08-07)
How to push this fork and merge upstream getAlby/hub into it. Every command
below was executed and verified on 2026-08-07 (fork at `b330fc3e`, merged 37
upstream commits, pushed to `welliv/alby-hub-greenlight`).
## Prerequisites
- `gh` authenticated as welliv (or `GITHUB_TOKEN` exported):
```bash
export GH_TOKEN=$(uv run python3 "$HOME/.hermes/skills/github/github-auth/scripts/git-credential-token.py")
gh auth status # → Logged in to github.com account welliv
```
- Local git identity pinned to the GitHub noreply address. **Required**:
welliv's account has email-protection enabled, so any push containing
commits authored with a non-associated email (e.g. the private
`jackal-botch-icon@duck.com`) is declined with
`push declined due to email privacy restrictions`.
```bash
git config user.name "welliv"
git config user.email "174869511+welliv@users.noreply.github.com"
```
If a push is declined anyway (older commits with the private email), rewrite
them before pushing:
```bash
export FILTER_BRANCH_SQUELCH_WARNING=1
git filter-branch -f --env-filter '
if [ "$GIT_AUTHOR_EMAIL" = "jackal-botch-icon@duck.com" ]; then
export GIT_AUTHOR_NAME="welliv"
export GIT_AUTHOR_EMAIL="174869511+welliv@users.noreply.github.com"
export GIT_COMMITTER_NAME="welliv"
export GIT_COMMITTER_EMAIL="174869511+welliv@users.noreply.github.com"
fi' -- <base-sha>..HEAD
# NOTE: rewrites every commit SHA in the range (trees unchanged); force-push after.
```
## Part A — Push the fork
```bash
git push origin feat/greenlight-backend
# if the local branch was rewritten: git push --force-with-lease origin feat/greenlight-backend
```
Verify the remote head matches local:
```bash
git rev-parse HEAD | cut -c1-8 # local tip
gh api repos/welliv/alby-hub-greenlight/branches/feat/greenlight-backend \
--jq '.commit.sha' | cut -c1-8 # remote tip
```
## Part B — Merge upstream
```bash
git remote add upstream https://github.com/getAlby/hub.git # once
git fetch upstream master
# Size the gap (should match the merge-base of the fork's own base):
git rev-list --count HEAD..upstream/master # commits behind
git merge-base HEAD upstream/master | cut -c1-12 # fork's base SHA
# Dry-run to see the conflicts before committing to anything:
git merge --no-commit --no-ff upstream/master
git diff --name-only --diff-filter=U # conflicted files
git merge --abort # clean up dry-run
```
### Resolving conflicts (2026-08-07: only 2, both frontend)
1. **`frontend/src/lib/backendType.ts` (modify/delete)** — upstream moved it to
`backendType.tsx` with a typed `backendTypeConfigs` record. Resolution:
`git rm` the old file and port the fork's `GREENLIGHT` entry into the new
`.tsx` (same shape: `hasMnemonic`, `hasChannelManagement`, `hasNodeBackup`).
2. **`frontend/src/screens/setup/SetupNode.tsx` (content)** — fork kept a local
display-config copy; upstream now renders from `backendTypeConfigs`. Take
upstream's version; the ported GREENLIGHT entry flows through automatically.
After resolving: `grep -rn "<<<<<<<" frontend/src/ --include="*.tsx" --include="*.ts"`
must return nothing.
## Part C — Verify the merge (all four gates)
```bash
export PATH=$PATH:/usr/local/go/bin
go build ./... # 1. Go compiles
go test ./lnclient/greenlight/ -count=1 -timeout 180s # 2. GL tests pass
cd frontend && yarn install --network-timeout 300000 # new upstream deps (e.g. qr-code-styling)
yarn build:http # 3. frontend compiles (tsc + vite)
cd .. && go build -o /tmp/hub-merged cmd/http/main.go # 4. live smoke (below)
```
Live smoke test against the regtest harness (preconfigured creds, fresh
workdir so env wins over the persisted config DB):
```bash
# fresh dir with .env: WORK_DIR, PORT, UNLOCK_PASSWORD, NETWORK=regtest,
# LN_BACKEND_TYPE=GREENLIGHT, GREENLIGHT_CREDS_PATH=/root/gl-harness/creds,
# GREENLIGHT_NODE_URI=localhost:<harness grpc port>
# setup (any valid BIP-39 mnemonic; preconfigured path ignores it) → start → check:
curl -s http://127.0.0.1:PORT/api/info # running: True, backend: GREENLIGHT
curl -s -X POST .../api/invoices -d '{"amountMsat":5000,...}' # invoice mints
```
## Part D — Commit & push the merge
```bash
git add -A
git commit -m "merge: upstream getAlby/hub master (N commits, <base>..<head>)
Conflict resolution (K files): ..."
git push origin feat/greenlight-backend
```
## Evidence record (2026-08-07 run)
| Step | Result |
|---|---|
| Commits behind upstream master | 37 (merge-base `bf9c346a9899`) |
| Conflicts | 2 (both frontend, see Part B) |
| `go build ./...` | OK |
| `go test ./lnclient/greenlight/` | ok (~1.5s, all pass) |
| `yarn build:http` | OK (dist 4.8MB; needed `yarn install` for `qr-code-styling`) |
| Live smoke (regtest harness) | running: True, GREENLIGHT, invoice minted, UI 200 |
| Post-merge drift vs upstream | **0** (relationship clean — see pitfall below) |
| Merge commit | `580b91e1` |
| Branch tip / pushed | `2d7529d1``+ b330fc3e...2d7529d1 (forced update)` ✓ |
## Pitfalls (learned the hard way)
- **Email privacy restriction**: welliv's account blocks pushes exposing the
private email. Always commit with the noreply address (Part Prereqs).
- **⚠️ NEVER use `git filter-branch` to rewrite the email of a merged branch**:
it rewrites *every* commit SHA in the range — including the upstream commits
— so the fork's history no longer matches upstream's SHAs and every future
`git merge upstream/master` re-conflicts on the same files
(`git rev-list --count HEAD..upstream/master` stays > 0 forever).
**Correct approach** (what this repo's history uses):
1. `git branch keep <rewritten-tip>` (save your commits)
2. `git reset --hard <fork-base>` (the last commit before the merge)
3. `git merge upstream/master` (brings **original** upstream SHAs) → resolve
conflicts
4. `git cherry-pick <fork-base>..keep^` (re-applies your local commits;
cherry-pick uses the current committer identity = noreply)
5. verify `git rev-list --count HEAD..upstream/master` → **0**
6. `git push --force-with-lease`
- **Persisted GL config**: `GREENLIGHT_CREDS_PATH`/`GREENLIGHT_NODE_URI` are
stored in the encrypted config DB at provisioning; env changes alone don't
take effect on an existing workdir. Use a fresh workdir to re-point a hub.
- **Frontend deps**: upstream adds deps (`qr-code-styling`); `yarn install`
before `yarn build:http` or tsc fails with TS2307.
- **Shallow clones can't serve as merge sources** — fetch upstream from GitHub
(https), not from a `--depth 1` local clone (no common ancestor).

View file

@ -1,41 +0,0 @@
# Mainnet readiness test plan — GL backend in Alby Hub
Gate: do each step in order. Do not start the next until the current one passes and evidence is recorded. Use small real amounts. This is the "rigor before real money" checklist.
## Phase 0 — Funded node on mainnet (the common pillar)
- Register a real Blockstream **mainnet** node (glcli `-n mainnet` register) with dev/staging certs; the network-map fix already covers `mainnet→bitcoin`.
- Point the Hub DB at it (`.env NETWORK=mainnet`, GreenlightNodeURI, CredsPath, SignerDataDir), restart, unlock.
- **Pass =** `/api/info` shows GREENLIGHT / mainnet / running; signer logs `-n mainnet`; `POST /api/invoice` returns a real `lnbc` (mainnet) BOLT11, pending.
- Gate: this phase must be stable across 3 restarts before anything else.
## Phase 1 — Funded real channel + small payment (SMALL: 1050k sats total)
- Fund the node's onchain address from a working mainnet faucet/self-send.
- Open a 1M5M sat channel to a healthy mainnet peer with routing.
- Issue an invoice on the peer; pay it through the Hub API (`POST /api/payments/:invoice`).
- **Pass =** tx in DB `outgoing / settled / <amount>` with a released **preimage**; balances reflect the route fee. This is the exact scenario I could NOT finish on testnet (no faucet) and must be proven on mainnet with real sats.
## Phase 2 — Backup / restore round-trip on a FUNDED mainnet node
- Trigger `/api/backup` → encrypted `.bkp` (note: it StopApp + db.Stop by design; API is down until `systemctl restart`).
- Wipe `.data`, restore via `/api/restore` (multipart: unlockPassword + backup file, NO jwt, setupCompleted=false) → 204.
- **Verify =** same node pubkey, same mnemonic, same onchain/channel balance, same invoice history. This is the data-loss insurance; it must be proven with money actually in the node.
## Phase 3 — Recovery under fault
- Kill the supervised signer mid-operation; confirm the supervisor restarts it without corrupting state.
- Confirm a crashed-while-paying tx does not coinbine/leak; the failure path returns a clean error.
- Confirm `ResetRouter` stub is never invoked in normal operation (it's a known intended stub from my audit; make sure no path calls it during regular use).
## Phase 4 — Security / release polish
- **Hardening already committed:** `.gitignore` now ignores `hub`, `.data.*`, `.env.bak*`; the network-mapping bug is fixed and committed (`535e66e`).
- Review the service is not exposed: confirm the HTTP/API port is not internet-reachable without auth (reference: routstrd :8008 bind lesson — never run the fund wallet reachable unbindfirewalled).
- Secret/seed material: no hsm_secret / GL creds committed (verify `git ls-files | grep -iE 'hsm|gfs|pem'` is empty).
- Run the full test suite green on the committed ref.
## Phase 5 — Regression on the release branch
- `MOVED ALL WORK: still ~20 modified + 5 untracked GL files. Before mainnet, decide: commit the remaining delta to the fork and open the PR against upstream getAlby/hub, OR keep on `feat/greenlight-backend`. At minimum the working tree should be committed so there's a reproducible build and rollback point.
## Explicit non-goals (do not slip in)
- No auto top-up / auto route logic changes here (separate feature).
- No signing/verify of the release binary yet — out of scope unless asked.
## Definition of done
Only when Phases 04 all pass with recorded evidence and the tree is committed/PR-d is the GL backend acceptable to run on mainnet. Until then: **dev/testnet/regtest only, small amounts, nothing internet-reachable without auth.**

View file

@ -127,3 +127,81 @@ layers of detection:
1. **Signer** (local, supervised, 15s respawn) — publishes `nwc_gl_signer_health`
2. **Node** (hosted, watchdog-probed, 30s interval + pump-stall) — publishes `nwc_gl_node_health`
3. **Hub** (degraded boot) — never refuses to start if the node is unreachable; serves `isReady:false` in the cached verdict
## Known issues
Status of each issue: **blocked** (upstream dependency), **mitigated**
(workaround shipped), or **fixed** (resolved + verified).
## 1. signmessage freezes the production signer — BLOCKED (upstream #739)
`signmessage` sends a signing request the production VLS signer cannot
answer; hsmd's queue wedges and the whole node freezes until Blockstream
restarts it. Reproduced twice (hosted testnet node A; regtest harness,
2026-08-08). Filed as [Blockstream/greenlight#739](https://github.com/Blockstream/greenlight/issues/739)
— unacknowledged.
- Mitigation: the backend stubs `SignMessage` (returns a clear error,
never forwards). The health watchdog (health.go) detects a freeze and the
hub boots degraded instead of locking the user out. RUNBOOK.md has the
recovery path.
- Unblocks when: Blockstream fixes the signer (or the issue is formally
acknowledged + a workaround documented).
## 2. Hold invoices — BLOCKED (stack-level)
Greenlight hosted nodes run no plugins and gl-client exposes no hold hooks
(verified by grep, 2026-08-08). `MakeHoldInvoice`/`Settle`/`Cancel` return a
clear error. Only feature gap vs LDK/CLN/LND. Do not ship a fake hold path.
## 3. Incoming keysend TLV records — FIXED (correction)
Earlier audits noted "TLVs dropped". Correction: `streamIncoming` maps the
incoming keysend `ExtraTLVs` into the transaction `Metadata` (`tlv_records`)
and they surface to NIP-47 clients via the freeform metadata blob. Verified
in source (streamincoming.go:87-96) + models.go comment.
## 4. Outgoing keysend preimage/hash recording — FIXED
CLN-family keysend derives its own preimage server-side; the caller's
preimage cannot be honored (no gRPC field). The backend now reports the
actual preimage + payment hash from the KeySend response and the
transactions service records those — the stored hash is the one that
actually settled (verified live on CLN + GL, 2026-08-08; upstream PR #2521).
## 5. ResetRouter — MITIGATED
The hosted node owns the network graph; there is no reset RPC. The stub
returns a clear error; the backup path tolerates it (warns + continues,
api/backup.go:63). Only the explicit reset-router API endpoint surfaces it.
## 6. Node hosted elsewhere — INHERENT (design)
The node's availability depends on Blockstream infrastructure (the flip
side of the custody split). The watchdog + runbook cover detection and
recovery; a frozen node cannot be restarted by the user.
## 7. channel-state notifications — MITIGATED
The GL node server leaves all six `cln.Node Subscribe*` stream RPCs
unimplemented (they panic the gl-plugin process). `nwc_channel_ready/closed`
notifications are not emitted; incoming payments are covered by the
restart-safe pump instead.
## 8. Signer seed at rest — PLAINTEXT (posture note, mainnet audit 2026-08-08)
The hub supervisor spawns `glcli signer run` directly, so the seed
(`hsm_secret`) sits plaintext in the 0700 data dir while the hub is
unlocked — the same posture as the hub's own LDK seed. An at-rest
encryption wrapper exists (`lnclient/greenlight/signer/encrypt-seed.py` +
`greenlight-signer-run.sh`, AES-256-GCM, for systemd deployments) but is
not wired into the supervisor. Wiring it is a follow-up; for a single-user
device deployment the 0700/0600 file perms match the rest of the wallet.
Audit-verified clean: dataDir 0700, hsm_secret/signer.log/signer.pid 0600,
device PEMs 0600, seed write-once (a different mnemonic can never desync
the signer from the node), signer liveness tracks the live process.
See [ARCHITECTURE.md](../lnclient/greenlight/ARCHITECTURE.md) for the
full design rationale (why we integrate through `glcli`, how GL primitives
map to the Go layer, and what operational additions we provide).

View file

@ -1,142 +1,68 @@
# Greenlight backend for Alby Hub
A dedicated `lnclient` backend that runs an Alby Hub against a
[Blockstream Greenlight](https://blockstream.com/greenlight/) node: a real
Core Lightning node hosted by Blockstream, with the signing keys held in a
separate VLS (Validating Lightning Signer) process.
A Core Lightning node on Blockstream's infrastructure. A VLS signer on your
Alby Hub. The two communicate over mTLS — Blockstream runs the node, your hub
holds the keys.
## What you get
## Why Greenlight for Alby Hub?
- Full NIP-47 surface: payments (pay_invoice, pay_keysend), invoices,
lookup, balances (channel + onchain), channels (open/close), onchain
(withdraw, new address), offers (BOLT12), peers, network graph.
(`sign_message` is NOT supported — see below.)
- Incoming payments: `WaitAnyInvoice` pump with a persisted pay index
(restart-safe catch-up, no missed payments) + `StreamIncoming` for
keysend TLV capture. Keysend custom records are surfaced in the
transaction Metadata under `tlv_records` and reach NIP-47 clients.
- Custody: the hub never holds the seed. All signing happens in the
signer process (VLS), which validates channel state and fees before
signing — a compromised hub cannot move funds.
Greenlight is the only backend where the node lives in the cloud but the keys
stay local. Every other backend bundles both together:
## Not supported (by design)
| | Node | Keys |
|---|---|---|
| LND | Self-hosted | Self-hosted |
| LDK | Embedded | Embedded |
| CLN | Self-hosted | Self-hosted |
| Bark | Cloud | Cloud (custodial) |
| **Greenlight** | **Cloud** | **Local** |
- Hold invoices (GL/cln node does not expose them)
- `sign_message`: the VLS signer (both the gl-testing python signer and
the production Rust glcli signer, verified on a live testnet node)
hangs on the hsmd SignMessage request, which wedges lightningd's serial
hsmd queue and freezes ALL signing operations (invoice creation times
out) until the hosted node restarts. The backend returns a clean
not-supported error instead and does not advertise the method.
- `ResetRouter` (LDK-only concept)
- Channel state notifications (`nwc_channel_ready` / `nwc_channel_closed`):
all six `cln.Node` `Subscribe*` stream RPCs are `unimplemented!()` stubs
on the GL node server — calling one **panics the gl-plugin process and
kills the node** (verified live). The backend deliberately never calls
them. `StreamNodeEvents` currently emits only `InvoicePaid`.
You get CLN's full feature set (bolt12 offers, keysend, plugins) without disk
management, uptime worry, or channel operations. Channels are provisioned
automatically by Blockstream's LSP. If the platform disappears tomorrow, the
same 32-byte seed that runs your signer boots a standalone CLN node — no
vendor lock-in.
## Setup
## Setup (two paths)
The GL backend supports two deployment paths:
**Product path** — the hub wizard registers a new node from a 12-word mnemonic,
extracts credentials, and starts the signer under supervision.
### Product path (hub wizard)
During setup, the hub calls `EnsureProvisioned` to register or recover a GL
node from the 12-word mnemonic, extract device credentials, and start the
signer under supervision. The signer runs as a subprocess supervised by the
hub (15s respawn, SIGTERM→KILL) and writes periodic backup snapshots to
`backup.json` (the GL-native signer backup — see [ARCHITECTURE.md](ARCHITECTURE.md)
for the full design).
### Manual path (glcli)
```sh
cargo install gl-cli
gl-cli register --network bitcoin # or recover an existing seed
# creates ~/.local/share/greenlight/{credentials.gfs, hsm_secret}
```
2. **Extract credentials** — the hub needs the device credentials as PEMs:
```sh
python3 lnclient/greenlight/extract_creds.py \
~/.local/share/greenlight/credentials.gfs ./greenlight-creds
# writes ca.pem, client.pem, client-key.pem, rune
# prints the node URI, e.g. gl1<node_id>.gl.blckstrm.com:443
```
3. **Run the signer** — separate process, encrypted seed:
```sh
python3 lnclient/greenlight/signer/encrypt-seed.py ~/.local/share/greenlight
# installs lnclient/greenlight/signer/greenlight-signer.service (or run
# greenlight-signer-run.sh by hand)
```
4. **Configure the hub**:
```sh
LN_BACKEND_TYPE=GREENLIGHT
GREENLIGHT_CREDS_PATH=/path/to/greenlight-creds
GREENLIGHT_NODE_URI=gl1<node_id>.gl.blckstrm.com:443
# optional:
GREENLIGHT_SERVER_NAME=gl1<node_id>.gl.blckstrm.com # TLS SNI, derived from URI if unset
GREENLIGHT_SIGNER_DATA_DIR=/home/greenlight/.local/share/greenlight # include seed in hub backups
```
## Architecture notes
- **Wire**: the node speaks the cln-grpc protocol (package `cln`, service
`cln.Node`) over one mTLS port. The backend reuses the hub's vendored
cln-grpc bindings (`lnclient/cln/clngrpc`), verified wire-compatible.
`StreamIncoming` lives on the `greenlight.Node` service (not in the
vendored bindings) and is consumed with a raw codec + protowire decoder
in `streamincoming.go`.
- **Reconcile**: the hub's invoice reconcile stays on until this backend
advertises `payment_received`; the pump's persisted index is what makes
that safe (see `waitanyinvoice.go`).
- **Sends**: synchronous `Xpay` (preimage + fees returned), same as the
CLN backend.
- **Duplicates**: invoice payments are published only by the pump;
`StreamIncoming` publishes keysends only (its invoice events are
ignored) — publishing both would double-create transactions.
## Testing
All functionality is tested against an in-process mock node (the vendored
`RegisterNodeServer` stub + a hand-registered `greenlight.Node` stream), so
the full suite runs without a live node or credentials:
**Preconfigured path** — point the hub at existing credentials and a node URI:
```sh
go test ./lnclient/greenlight/ -v
LN_BACKEND_TYPE=GREENLIGHT
GREENLIGHT_CREDS_PATH=/path/to/greenlight-creds
GREENLIGHT_NODE_URI=gl1<node_id>.nodes.gl.blckstrm.com:443
```
### Live E2E (real node)
## Status
`live_e2e_test.go` runs against a real GL stack and is skipped unless
`GREENLIGHT_LIVE=1` plus the standard env vars are set. It has been
executed end-to-end against a fully local Greenlight stack (real
`lightningd` + real `gl-plugin` gRPC + real VLS signerproxy, the same
binaries Blockstream runs), proving:
30 of 35 LNClient methods implemented via standard CLN gRPC. 4 are honest
`ErrUnsupported` (hold invoices, signmessage — architecturally blocked). 1
stub. Tested live on Blockstream testnet: invoices created, addresses
generated, balances reported, backup/restore proven.
- **production lightningd version is `v25.05gl1`** (readable only via a raw
`Getinfo` — the hub's `NodeInfo` doesn't expose it);
- **`WaitAnyInvoice` timeout semantics**: with no pending payments it
returns RPC error 904 `Timed out while waiting for invoice to be paid`
after the requested timeout — the pump treats this as "nothing new" and
keeps polling (no error spam, no missed payments);
- **`StreamIncoming`** establishes and stays open (no immediate close),
confirming the raw-codec zero-codegen consumer works against the real
`greenlight.Node` service;
- real invoices created/queried, real balances/peers/graph listed.
Keysend and receive subtests are conditional on channel balance.
## What we added on top
To reproduce on a box without a Blockstream node, run the gl-testing local
stack (requires `bitcoind`, `cfssl`, `cfssljson`, `cln-version-manager`,
and `cargo build -p gl-plugin -p gl-signerproxy` in a clone of
`Blockstream/greenlight`), then a harness test that registers a client,
starts the node, and exports `credentials.gfs` + `ca.pem` (see
`libs/gl-testing/tests/test_live_harness.py`). Note the gl-testing
in-process python signer hangs on `signmessage` (freezing the node), so
`SignMessage` is skipped in live mode; the production signer is the Rust
VLS binary, a different implementation.
Greenlight provides the signer and node. We added what a hub needs to run
it as a service:
- **Signer supervisor** — 15s respawn, SIGTERM→KILL, Signal(0) liveness,
health surfaced in `GetNodeStatus`
- **Health watchdog** — periodic Getinfo, pump-stall detection, degraded boot
(a frozen node doesn't lock you out of your hub)
- **WaitAnyInvoice pump** — persisted pay index, busy-loop protection
- **Provisioning** — mnemonic→seed→register→extract creds→launch signer,
fully automated
## Backups and exit strategy
The signer writes `backup.json` when channels exist. The hub `.bkp` includes
it. If Blockstream's cloud goes away: `hsm_secret` → CLN node identity,
`backup.json``glcli signer convert-backup --format cln` → SCB →
`lightning-cli recoverchannel`. Everything needed to self-host is already on
your disk.
[ARCHITECTURE.md](ARCHITECTURE.md) covers the integration design.
[docs/RUNBOOK.md](docs/RUNBOOK.md) covers operations and troubleshooting.