cln/plugins/bkpr
Rusty Russell 1dda0c0753 bookkeeper: don't flood logs if we have many channelmoves all at once.
Since we're synchronous, these only reach lightningd after we're done:
in the case of 1.6M channelmoves, that can give it major heartburn.

In practice, this reduces the first bkpr command on a fresh upgrade
from 349 to 235 seconds (but this was before other improvements we did
this release).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `bookkeeper` reduced logging for large imports to increase speed.
2025-11-04 12:47:37 +10:30
..
test Makefile: create a library containing common, wire and bitcoin objects. 2025-10-23 06:44:04 +10:30
account.c common: reintroduce "ignored" primary tag. 2025-08-28 13:41:44 +09:30
account.h ]bkpr: save accounts in the datastore instead of database. 2025-08-19 13:37:50 +09:30
account_entry.c bkpr: strings in structures should be const. 2025-08-19 13:37:50 +09:30
account_entry.h bkpr: strings in structures should be const. 2025-08-19 13:37:50 +09:30
blockheights.c bookkeeper: fix restoration of derived wallet blockheights on restart. 2025-11-03 12:36:38 +10:30
blockheights.h bkpr: add in-mem & datastore storage for external blockheights. 2025-08-19 13:37:50 +09:30
bookkeeper.c bookkeeper: don't flood logs if we have many channelmoves all at once. 2025-11-04 12:47:37 +10:30
bookkeeper.h lightningd: migrate events from bookkeeper at startup. 2025-08-19 13:37:50 +09:30
chain_event.c bkpr: use the in-memory event descriptions, not the database ones. 2025-08-19 13:37:50 +09:30
chain_event.h bkpr: forward utxo_deposit / utxo_spend notifications to new injectutxodeposit / injectutxospend calls. 2025-08-19 13:37:50 +09:30
channel_event.c bkpr: take, don't steal in new_channel_event. 2025-08-19 13:37:50 +09:30
channel_event.h bkpr: take, don't steal in new_channel_event. 2025-08-19 13:37:50 +09:30
channelsapy.c bkpr: don't leak event in fillin_apy_acct_details. 2025-08-28 13:42:04 +09:30
channelsapy.h bkpr: convert all the local db sql queries into calls to sql plugin. 2025-08-19 13:37:50 +09:30
descriptions.c bookkeeper: infrastructure to keep descriptions (annotations) in mem and datastore. 2025-08-19 13:37:50 +09:30
descriptions.h bookkeeper: infrastructure to keep descriptions (annotations) in mem and datastore. 2025-08-19 13:37:50 +09:30
incomestmt.c bookkeeper: honor start and ent times when consolidating onchain fees. 2025-10-21 20:21:56 +10:30
incomestmt.h bkpr: convert all the local db sql queries into calls to sql plugin. 2025-08-19 13:37:50 +09:30
Makefile Makefile: create a library containing common, wire and bitcoin objects. 2025-10-23 06:44:04 +10:30
onchain_fee.c bookkeeper: honor start and ent times when consolidating onchain fees. 2025-10-21 20:21:56 +10:30
onchain_fee.h bookkeeper: honor start and ent times when consolidating onchain fees. 2025-10-21 20:21:56 +10:30
README.md chore: fix some minor issues 2025-08-15 11:05:51 +09:30
rebalances.c global: remove unnecessary includes from headers. 2025-10-23 06:44:04 +10:30
rebalances.h bkpr: infrastructure to keep rebalance pairs in mem & datastore. 2025-08-19 13:37:50 +09:30
recorder.c bookkeeper: remove database. 2025-08-19 13:37:50 +09:30
recorder.h bookkeeper: save last timestamp to avoid another query in find_consolidated_fees. 2025-10-21 20:21:56 +10:30
sql.c bkpr: helpers to query sql plugin for chainmoves and channelmoves. 2025-08-19 13:37:50 +09:30
sql.h bkpr: restore run-recorder. 2025-08-19 13:37:50 +09:30

The bookkeeper keeps track of coins moving through your Lightning node.

See the doc/PLUGINS.md#coin_movement section on the message that CLN emits for us to process.

// FIXME: add more detailed documenation for how bookkeeper works.

3rd Party Coin Movements

Bookeeper ingests 3rd party plugin notifications about on-chain movements that it should watch.

This allows for us to account for non-internal on-chain wallets in the single place, making bookkeeper your single source of truth for bitcoin for an organization or node-operator.

As a plugin writer, if you want to emit onchain events that the bookkeeper should track, you should emit an event with the following format:

{
	"utxo_deposit": {
		"account": "nifty's secret stash",
		"transfer_from: null,
		"outpoint": xxxx:x,
		"amount_msat": "10000sat",
		"coin_type": "bc",
		"timestamp": xxxx,
		"blockheight": xxx,
	}
}
{
	"utxo_spend": {
		"account": "nifty's secret stash",
		"outpoint": xxxx:x,
		"spending_txid": xxxx,
		"amount_msat": "10000sat",
		"coin_type": "bc",
		"timestamp": xxxx,
		"blockheight": xxx,
	}
}

Withdrawing money (sending to a external account)

Sending money to an external account is a bit unintuitive in the UTXO model that we're using to track coin moves; technically a send to an external account is a "deposit" to 3rd party's UTXO.

To account for these, bookkeeper expects to receive a utxo_deposit event for the creation of an output to a 3rd party. It's assumed that you'll issue these at transaction creation time, and that they won't be final until we receive notice of spend of the inputs in the tx that created them.

To notify that money is being sent to a 3rd party output, here's the event we'd expect.

The two keys here are the following:

  • The account is external. This is a special account in bookkeeper and used for tracking external deposits (aka sends)
  • The transfer_from field is set to the name of the account that is sending out the money.
{
	"utxo_deposit": {
		"account": "external",
		"transfer_from": "nifty's secret stash",
		"outpoint": xxxx:x,
		"amount_msat": "10000sat",
		"coin_type": "bc",
		"timestamp": xxxx,
		"blockheight": xxx,
	}
}

List of todos

List of things to check/work on, as a todo list.

  • Transfers btw a 3rd party wallet and the internal CLN wallet? These should be registered as internal transfers and not show up in listincome