The plugin start command only accepted plugin options flattened onto
the RPC call (e.g. via -k/--keyword), but the plugin RPC schema
documents an explicit 'options' array. This mismatch meant that callers
using named parameters against generated RPC bindings (cln-rpc, grpc,
protobuf), which cannot flatten arbitrary options, failed with
'unknown parameter options'.
Options without a value are treated as boolean flags, matching the
flattened form.
Changelog-Fixed: JSON-RPC: `plugin start` now accepts plugin options as an `options` array of `keyword=value` strings, as documented in the `plugin` schema.
We need to wait for bwatch to see the deposit block or it won't see a
reorg.
```
assert l1.db_query('SELECT COUNT(*) AS c FROM outputs')[0]['c'] == 1
# Reorg the deposit block away. Deprioritize the returned mempool tx
# (same trick as simple_reorg) so the replacement blocks don't just
# re-confirm it.
bitcoind.rpc.invalidateblock(bitcoind.rpc.getblockhash(deposit_height))
memp = bitcoind.rpc.getrawmempool()
assert txid in memp
for t in memp:
bitcoind.rpc.prioritisetransaction(t, None, -1000000)
bitcoind.generate_block(2)
> l1.daemon.wait_for_log(r'Reorg detected', timeout=60)
```
Changelog-None
When an important plugin dies, lightningd shuts down while other plugins
may still be mid-sync-RPC during init. If rpc_open() fails because the
RPC socket is already gone (logging "Could not connect ... Connection
refused"), sync_req() proceeds with sync_fd == -1, and the subsequent
read(-1) fails with EBADF, producing the BROKEN message:
Reading sync lightningd: Bad file descriptor
This is the same intentional shutdown that already produces the
whitelisted "Reading sync lightningd: Connection reset by peer" (or a
clean EOF exit); the errno merely differs by the connection race. Add
"Bad file descriptor" to the broken_log whitelist so the test no longer
fails at teardown on this race.
Changelog-None
In particular, "struct jsonrpc_request"'s id is always a string.
cmd->id isn't, though.
We can also remove the now-unused json_get_id.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
An inline plugin's Plugin() object lives in the test process, and
Plugin() installs a PluginLogHandler on the root logger so that a
plugin author's logging calls reach lightningd. In the test process
that handler also forwards pyln's own machinery logs into the node's
log whenever the test-side logger emits at DEBUG. In particular
wait_for_logs() announces 'Waiting for [pattern]' with the pattern
embedded verbatim, so the announcement lands in the very log being
scanned and matches itself, silently reducing the wait to a no-op.
That let test_sendpay_notifications_nowaiter race its channel close
against the first payment (both payments then fail and the success
assertion sees an empty list); any literal-pattern wait_for_log on an
inline-plugin node is similarly voided under DEBUG logging.
Attach a filter to the inline plugin's log handler that only forwards
records originating outside the pyln packages, so author logging still
reaches the node log but pyln internals never do.
The new test covers both directions: an author log line is found by
wait_for_log, and a never-logged sentinel genuinely times out (it
matched instantly via the forwarded announcement before this fix).
The node directory embeds the test name, and this test's long name
pushes inline-plugin.sock past the AF_UNIX bind cap on some runs:
Linux's 108 bytes under liquid-regtest's longer network dir, and
Darwin's 104 with even the default path. Teach _inline_plugin() to
fall back to binding through a short symlink alias to the socket's
directory -- the same technique UnixSocket.connect already uses on
Darwin, aliasing the directory rather than the socket since bind
can't traverse a dangling final-component symlink. The socket file
still lands where the shim's cwd-relative connect expects it. The
test runs in the macOS smoke set too, so CI exercises the fallback on
the platform with the tighter cap.
Fixes: #9343
Changelog-None
The wallet registers its own scriptpubkey watches at node startup
since the bwatch series landed. The test anticipates them by
recording a total-count baseline first, but on a slow machine the
registration can land after the baseline is taken: an ASan CI run
caught a baseline of 0 and a final count of 105. A total-count
assertion races background registration no matter when the baseline
is read.
Count only the watches this test adds (their identifiers are
distinctive), which no background registration can perturb. The
test's per-watch assertions already work that way.
Fixes: #9360
Changelog-None
The test mines ~106 blocks (channel close, replacement channel
funding, then generate_block(99)) immediately before making three
payments, without waiting for the nodes to process those blocks.
Under valgrind the paying node can lag far behind bitcoind when the
payments start, so it builds onions whose cltv expiries the caught-up
peers reject. Two CI failures showed both shapes of the race:
In one run l1 was ~40 blocks behind (l2/l3 at block 210, l1 still
catching up), so l2 failed the forward: "Expiry cltv 181 too close to
current 210" (expiry_too_soon). Since the error came from the
invoice's route-hint channel, xpay disabled it and the payment failed
hard: "All 1 channels to the destination are disabled."
In the other run l1 was only a few blocks behind, so l3 failed the
final hop: "Expiry cltv too soon 211 < 210 + 5". xpay's
waitblockheight retry then succeeded, but the failed first attempt
left status 'failed' in the forwards row the test later asserts is
'settled'.
The test paid with pay until 7f6a33394 ("pytest: use xpay, not pay in
misc tests."); pay took its start height from bitcoind's headercount
and tracked the node's chainlag, so a lagging node still built onions
valid at the true chain tip. xpay uses its own notification-fed
blockheight plus one block of slack, which is why this only started
flaking recently.
Sync the nodes' blockheights before paying, as other tests do after
mining bursts.
Changelog-None
These tests built their RawProxy against a hardcoded bitcoin.conf in the node's datadir. That only worked on liquid-regtest by accident: BitcoinD.__init__ used to write bitcoin.conf even when the node was actually an ElementsD. Now that config writing is deferred to set_port() and ElementsD only writes elements.conf, the mocks found no config and no .cookie, so their RPC calls failed and bcli's getblock timed out, leaving lightningd stuck during restart.
Use bitcoind.conf_file, which points at the config the running daemon actually uses (elements.conf on liquid-regtest, bitcoin.conf on egtest).
The test mined a block and then reorged it, but it did not wait until bwatch had actually seen that block first.
Sometimes bwatch was still one block behind. In that case, the replacement
block looked like a normal new block, not a reorg, so the test waited forever for "Reorg detected".
Wait until bwatch has stored the block before reorging it.
Replace non-standard node variable names (n, n2, node, ln) with l1/l2
across test_plugin.py and test_misc.py. Convert four tests in
test_connection.py to use get_nodes() instead of consecutive get_node()
calls with identical options (slightly faster).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reduces the number of misc test plugins.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Also made fields not go optional if they are deprecated in the case that their parent is also deprecated. This makes more sense imo when you deprecate a whole command.
I added deprecation annotation for the .proto file as well so that later in the convert.rs file it properly annotates everything to allow for deprecated stuff.
Also fixed one propagation bug for the added/deprecated fields, that's why there are some additional changes of some fields.
Changelog-None
The plugin still loads (so its options and RPCs stay registered) but
stays inert — no chain polling, no watch replay — unless the user
explicitly opts in with --experimental-bwatch.
I'm not entirely sure about changing this test but this test was written prior to bwatch. Bwatch unconditionally triggers datastore writes on startup when it polls bcli.
If the async poll chain completes during shutdown rather than normal operation, the db_write hook fires after dblog has wound down, so the main DB gets the write but dblog doesn't capture it.
Changelog-Deprecated: JSON-RPC: `keysend` (use `xkeysend`). `xkeysend` will take over `keysend` in v27.03.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Use xpay and listpays.
Some tests which are pay/paystatus specific simply enabled deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `pay` and `paystatus`: use `xpay`, `listpays` (or `xpay`'s notifications for details of attempts). `pay` will be replaced by `xpay` in v27.03.
If you don't use complex pay options, you can use `invstring` now and when xpay
becomes pay, there's no transition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `pay` now accepts `invstring` as a parameter name for `bolt11`, to ease transition when xpay takes over in v27.03.
The main change here is the error messages got better. Where that was
the case, and the test was not completely redundant with existing xpay
tests, I explicitly converted to calls to xpay (meaning the tests
will be kept when `pay` is finally removed).
Test changes:
1. deschash checking is old-pay only, keep that there.
2. Add debug logs for checks for using forwarding via scid in injectpaymentonion.
3. Don't assert that not all sendpays should have the invoice string: xpay does not try to be clever there.
4. We no longer check if invoices are already paid before routing, since injectpaymentonion will catch that for us.
5. Removed `test_pay_get_error_with_update` which has xpay equiv.
Changelog-Changed: JSON-RPC: `xpay` now handles `pay` command by default (use `xpay-handle-pay=false` to prevent this)
Changelog-Removed: JSON-RPC: `exclude` parameter to `pay` (when `xpay-handle-pay` is True): craft a layer with desired modifications and pass it to `xpay` `layers`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This usually means changing the error strings, where we test them, and
avoiding `paystatus`.
Other changes:
1. Completely get rid of `test_custom_notification_topics`: xpay tests do this
already.
2. Add xpay wrapper to pyln-client so we can mix named and unnamed args.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We add a nice getroutes wrapper and use it to replace the simple
getroute() calls which simply hand the result to sendpay().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Touches a pile of tests, but they're easy to find.
Changelog-Changed: Protocol: We now wait 72 blocks, not 12, before closing channels (BOLT update)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I've reworked this: in general we should clear the refresh bit before
calling the jsonrpc to do the update. This allows the wait callback to
set the bit again if there's more to do, so we won't lose entries.
Now it's clear that we can remove the overzealous assert.
sql: plugins/sql.c:1749: refresh_by_created_index: Assertion `td->refresh_needs != REFRESH_UNNECESSARY' failed.
sql: FATAL SIGNAL 6 (version v26.04rc2)
0x5618e147892e send_backtrace
common/daemon.c:38
0x5618e14789bb crashdump
common/daemon.c:83
0x7f54d10ea04f ???
./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f54d1138eec __pthread_kill_implementation
./nptl/pthread_kill.c:44
0x7f54d10e9fb1 __GI_raise
../sysdeps/posix/raise.c:26
0x7f54d10d4471 __GI_abort
./stdlib/abort.c:79
0x7f54d10d4394 __assert_fail_base
./assert/assert.c:94
0x7f54d10e2ec1 __GI___assert_fail
./assert/assert.c:103
0x5618e1472725 refresh_by_created_index
plugins/sql.c:1749
0x5618e14736af one_refresh_done
plugins/sql.c:579
0x5618e1473932 limited_list_done
plugins/sql.c:1738
0x5618e1477418 handle_rpc_reply
plugins/libplugin.c:1093
0x5618e1477548 rpc_conn_read_response
plugins/libplugin.c:1398
0x5618e147ec71 next_plan
ccan/ccan/io/io.c:60
0x5618e147ef90 do_plan
ccan/ccan/io/io.c:422
0x5618e147f049 io_ready
ccan/ccan/io/io.c:439
0x5618e147ffae io_loop
ccan/ccan/io/poll.c:470
0x5618e14786af plugin_main
plugins/libplugin.c:2461
0x5618e1474b12 main
plugins/sql.c:2219
0x7f54d10d5249 __libc_start_call_main
../sysdeps/nptl/libc_start_call_main.h:58
0x7f54d10d5304 __libc_start_main_impl
../csu/libc-start.c:360
0x5618e1470710 ???
_start+0x20:0
0xffffffffffffffff ???
???:0
Diagnosed-by: Lagrang3 <lagrang3@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: Introduced this release.
Updating channelmoves table makes sql to crash if there are more
elements in listchannelmoves than limit_per_list.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
For these, when things change, we simply delete amd recreate the
changed entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `sql` plugin tables "htlcs", "forwards", "invoices", "sendpays" and "networkevents" are now updated more efficiently.
This is trivial now, as the invoice table gets reloaded every time,
but is an important check as we improve the implementation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Decryption is pretty efficient, but incoming traffic can bog down
connectd, especially on smaller nodes, so simply limit it to 1MB per
second.
This triggers in various tests, which is good: shows that it's working,
and that we continue to (slowly!) process traffic.
Changelog-Fixed: connectd: throttle incoming peers to give fairer peer handling under stress.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Replay of stored blinded HTLCs can reach onion decoding before the
hsmd-backed ecdh() wrapper is initialized, causing a startup crash.
Set up ECDH earlier and add a regression test for blinded HTLC replay.
Changelog-Fixed: lightningd no longer crashes when replaying stored blinded HTLCs during startup.
Adds the preimage to the forward_event notification when status is
settled. This can be useful to gather some context when listening to
this event
Changelog-Changed: forward_event notification no has preimage set if
status is settled.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: support for options which accumulate if specified more than once ("multi": true).
xpay can get upset if askrene goes away first:
lightningd-1 2026-02-18T02:47:44.908Z **BROKEN** plugin-cln-xpay: askrene-create-layer failed with {"code":-32601,"message":"Unknown command 'askrene-create-layer'"}
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add `test_bcli_concurrent` to verify bcli handles concurrent requests while the `getblockfrompeer` retry path is active, simulating a pruned node scenario where `getblock` initially fails.
Add `test_bcli_retry_timeout` to verify lightningd crashes with a clear error message when we run out of `getblock` retries.
It's always true for the first hook invocation, but if there is more
than one plugin, it could vanish between the two! In the default configuration, this can't happen.
This bug has been around since v23.02.
Note: we always tell all the plugins about the peer, even if it's
already gone.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: possible crash when peers disconnected if there was more than one plugin servicing the `peer_connected` hook.
Reported-by: https://github.com/santyr
Fixes: https://github.com/ElementsProject/lightning/issues/8858
Avoids guessing what the timeout should be, use a file trigger. This
is more optimal, and should reduce a flake in test_sql under valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It uses the hold_invoice plugin to ensure that an HTLC is in flight, but
it tells it to hold the HTLC for "TIMEOUT * 2" which is a big number under CI.
Reduce it to sqrt(TIMEOUT + 1) * 2, which works for local testing (I run
with TIMEOUT=10) and still should be enough for CI (TIMEOUT=180).
Christian reported that the test took 763.00s (!!) under CI.
On my build machine (TIMEOUT=90):
Before:
383.00s
After:
64.38s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If l2 hasn't seen l1's node_announcement yet:
```
# Correctly handles missing object.
> assert l2.rpc.sql("SELECT option_will_fund_lease_fee_base_msat,"
" option_will_fund_lease_fee_basis,"
" option_will_fund_funding_weight,"
" option_will_fund_channel_fee_max_base_msat,"
" option_will_fund_channel_fee_max_proportional_thousandths,"
" option_will_fund_compact_lease"
" FROM nodes WHERE HEX(nodeid) = '{}';".format(l1.info['id'].upper())) == {'rows': [[None] * 6]}
E AssertionError: assert {'rows': []} == {'rows': [[None, None, None, None, None, None]]}
E
E Differing items:
E {'rows': []} != {'rows': [[None, None, None, None, None, None]]}
E
E Full diff:
E {
E - 'rows': [
E + 'rows': [],
E ? ++
E - [
E - None,
E - None,
E - None,
E - None,
E - None,
E - None,
E - ],
E - ],
E }
tests/test_plugin.py:4131: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are explicitly written to check that the values don't accidentally change,
which applies to both old and new styles.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means:
1. downgrade changes (we no longer fail due to node biases).
2. various deprecations no longer are
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because l1 and l3 allow localhost as a broadcastable address, they can
try to reconnect. Disable reconnections, so we don't race:
```
> l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
tests/test_plugin.py:4146:
...
elif "error" in resp:
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'host': 'localhost', 'port': 45035}, error: {'code': 402, 'message': 'disconnected during connection'}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's one more complaint we can see when plugins get upset:
```
lightningd-1 2026-01-12T06:10:49.317Z **BROKEN** plugin-cln-xpay: askrene-create-layer failed with {"code":-4, "message":"Plugin terminated before replying to RPC call."}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not reliable:
```
# We should have deferred hook update at least once!
> l2.daemon.wait_for_log("UNUSUAL plugin-dep_b.py: Deferring registration of hook htlc_accepted until it's not in use.")
tests/test_plugin.py:2646:
...
if self.is_in_log(r):
print("({} was previously in logs!)".format(r))
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile("UNUSUAL plugin-dep_b.py: Deferring registration of hook htlc_accepted until it's not in use.")]" in logs.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
