We add regular channel event pruning, as otherwise the database may get
filled quickly. We add two mechanisms, a retention time and a max events
number. Both can be turned on individually.
Add a PruneChannelEvents query that bounds the channel_events table by both
size and age in a single statement: an id-keyset offset enforces a maximum
event count and a timestamp filter enforces a retention window, OR-joined so
each limit applies independently. The query returns the number of rows deleted
so callers can surface pruning activity.
Add a standalone timestamp index so the global age-based prune does not scan
the full table. The existing composite index leads with channel_id and cannot
serve a channel-agnostic timestamp filter.
Bump the integration test lnd binary to v0.21.0-beta. Two test fixes are
required for the new version:
- nodereport: the anchor commitment close fee changed by 10 sat, update the
hardcoded CHANNEL_CLOSE_FEE expectation from 4535 to 4525 sat.
- test_context: closeChannel mined a block while still waiting for the pending
close update, which confirmed the force close tx out of the mempool before
its fee could be read, causing a 'Transaction not in mempool' failure. Only
start mining once the close fee has been captured from the mempool.
Open a channel, seed forwarding events with payments, and assert the
ForwardingAbility RPC returns a decodable response whose window matches the
request. Note that we can only simulate forwards back to the same node
because we only have two lnd nodes available in tests.
Add the frcli forwardingability command to query the RPC over a time range
and optional liquidity floor. It decodes the sparse response and prints each
pair with its raw effective uptime and forwarded volume, plus uptime fraction
and velocity derived from the reported window.
Encode the per-pair forwarding abilities into the packed, deduplicated wire
form and decode them back. Only pairs with non-zero effective uptime or
forwarded volume are emitted; the window the metrics cover is carried on the
response so consumers can derive uptime fraction and velocity themselves.
Extend the forwarding analyzer to produce a ForwardingAbility per peer pair:
the effective uptime (time the pair held at least the liquidity floor of
directional forwardable liquidity) and the total forwarded amount, both as
raw facts with no derived rates or categories. A single uniform floor is
applied to every pair so effective uptime is comparable across pairs, and
forwarded volume is reported even when uptime is zero so consumers keep the
demand signal.
The percentile threshold model and its quantile helper are removed in favour
of the single floor.
Fetch lnd's forwarding history in paginated batches rather than a single
call, so the analysis is not silently truncated at lnd's default page size.
The stub forwarding client returns an empty page for non-zero offsets so the
pagination loop terminates in tests.
Add the ForwardingAbility RPC that reports, for every (peerIn, peerOut)
pair, the raw forwarding facts over a window: effective uptime in seconds
and forwarded volume in satoshis. The response is a sparse, packed encoding
(deduplicated peer keys plus packed pair indices) carrying only pairs with a
non-zero signal; an absent pair means zero over the window. A single
liquidity_floor_sat request parameter sets the directional liquidity a pair
must hold to count as economically forwardable.
loadPeerEvents fetched an entire channel's events in a single
math.MaxInt32-limited query. Page through the store in id-ascending
batches so no single query is unbounded, and document the paging
protocol on the EventsSource contract.
Promote the existing test-store and test-DB constructors from
*testing.T to testing.TB so the upcoming EffectiveUptime benchmark can
share the same fixture path as the existing tests. testing.TB is the
shared interface of *testing.T and *testing.B, so every current caller
keeps type-checking unchanged.
Wire EffectiveUptime as the analyzer's public entry. The pipeline
resolves channels to peers from the store, folds lnd's forwarding
history into per-pair success amounts, augments the considered set with
lnd's open and closed channels to hedge survivorship bias, seeds each
channel's state at startTime, and dispatches every peer pair to the
bidirectional walk.
calculateAllPairsUptime walks the unordered cross-product (i, j>=i) with
a lazy per-peer event cache. Each peer's events are fetched once and
replayed across every pair that consumes them. The cached events live as
a slice so the cross-pair merge can use a two-pointer walk instead of an
iter-based merge that would need goroutine plus channel synchronisation
per event.
Establish the boundary between the chanevents store and the upcoming
forwarding analyzer plus the seed walk every pair calculation depends
on. EventsSource is the read surface the analyzer consumes, and
ForwardingAnalyzer carries it alongside an lnd handle so the upcoming
driver can fold lnd's open and closed channel sets into the considered
population.
getInitialChannelState reconstructs a channel's state at startTime by
seeding from the latest pre-window update and replaying any residual
same-timestamp siblings the SQL keyset may have surfaced. The residual
range is bounded by definition (events between two adjacent timestamps
within a channel), so streaming would buy nothing over materialising
the slice in one call. A guard error flags later-timestamp updates in
the residual walk as schema drift.
The per-pair uptime walk derives forwarding ability for both (A→B) and
(B→A) directions in a single chronological pass over the merged event
stream. Two independent state copies, one rooted at each peer,
accumulate each direction's uptime against its own balance threshold.
A liquidity check that would otherwise be O(channels-per-peer) per tick
becomes O(1) via four running balance sums that shadow each side's
online inbound and outbound capacity. The merge loop adjusts these sums
as events mutate channel state, and the per-tick threshold check reads
the mins inline.
Add Quantile, a generic linear-interpolation q-quantile over a slice of
sortable numeric values. The forwarding-ability analyzer needs to
characterise the distribution of historical forwarded amounts and uses a
configurable percentile as the headline statistic; lifting the
computation into its own helper keeps the analyzer focused on forwarding
logic and gives the quantile contract its own table-driven test that
covers the interpolation rule and the empty/out-of-bounds error paths.
Add GetChannelByShortChanID, the inverse of AddChannel. The
forwarding-ability analyzer receives scids from lnd's forwarding history
and must map them back to the chanevents store's internal channel id to
query events.
Add ScidToPeerMap, which materialises a snapshot of every short channel
id paired with the pubkey of the channel's remote peer. Forwarding-data
sources index events by short channel id, but downstream analyses need
to attribute behaviour to the peer, not the channel. The map skips
channels whose short channel id is still zero (unconfirmed), so callers
see only fully advertised channels. Coverage extends TestStore with a
two-channel fixture pinning the join.
Add GetLatestChannelUpdateBefore, which fetches the most recent
EventTypeUpdate strictly before a given instant. Forwarding-ability
analyses that summarise behaviour over a window must seed their state
from the channel's balance at the window's lower bound; without the
ability to look back past that bound, the first events in the window
have no baseline to compare against. The lookup tolerates missing
predecessors by returning (nil, nil), letting callers distinguish "no
prior update" from a genuine error. Coverage exercises both the present
and absent cases against the existing TestStore fixture.
Adds a `chanevents` subcommand that wraps GetChannelEvents and exposes
chan_point, start/end time, max_events, and last_id flags. The help
text documents the manual pagination contract: keep re-running with
--last_id set to the previous response's last_id until has_more is
false, while leaving --start_time and --end_time fixed across calls.
Drives a regtest channel through open, payment, and force-close and
asserts that GetChannelEvents surfaces the expected mix of online,
offline, and balance-update events. The same window is then walked
with a small page size to verify last_id round-trip, has_more
termination, and MaxEvents clamping in one pass.
Implements the handler against the chanevents store. A zero end_time
defaults to the server's current wall clock so callers can omit it for
"up to now" queries. max_events is clamped to a 10000-row hard cap that
also serves as the implicit default when the caller leaves the field at
zero. An unknown chan_point maps to NotFound; negative time bounds and
start_time after end_time map to InvalidArgument. The response sets
has_more whenever the page filled to the requested limit so the client
knows to keep paginating.
Also registers the new endpoint in the macaroon permissions table
under the channels:read entitlement.
Threads the chanevents.Store the daemon already constructs into the
frdrpcserver.Config so the upcoming GetChannelEvents handler has a
read path. Both standalone and subserver startup wire the same store
instance.
Adds the GetChannelEvents RPC to the proto and regenerates the gRPC,
gateway, swagger, and JSON stubs. The request carries a chan_point,
inclusive start_time and exclusive end_time bounds, a max_events cap,
and a last_id keyset cursor; the response echoes last_id and a has_more
flag so callers can drive pagination without server-side state.
Switches the GetChannelEvents query from a timestamp-ordered scan to an
id-keyset cursor (WHERE id > $cursor ORDER BY id ASC LIMIT $n). The
keyset cursor is stable under concurrent inserts and survives a future
retention job that prunes the oldest rows: a positional OFFSET would
silently skip events whenever rows below the cursor are deleted, while
"id > $cursor" keeps advancing past whatever the caller has already
seen. The id field is documented in the proto as a server-assigned
monotonic identity, so callers persist last_id as their sync watermark.
Adds a (channel_id, id) composite index to back the new query; the
existing (channel_id, timestamp) index does not cover it and would
force a per-channel filter after a global id scan.
We want to know if an update came from an initial sync. This also helps
us to identify data gaps and one can be sure it was not due to an actual
event. Modify the migration as it's unreleased.
