[log] Introduce log rate limiter class

The LogRatelimiter class implements a fixed window rate limiter. The
rate limiter allows a fixed amount of bytes to be consumed within a
fixed time window.

[log] Introduce source location type

The SourceLocation type stores the filename and line of a source code
location.
In a later commit we use this type as the key type in an unordered map
and set to keep track of rate limters for each location.

[config] Add -ratelimitlogging config option

The -ratelimitlogging can be used to enable/disable the rate limiting to
disk. Rate limiting is enabled by default.

[log] Add two new categories for unconditional logging

We create two new categories `UNCONDITIONAL_ALWAYS` and
`UNCONDITIONAL_RATE_LIMITED` that are always enabled by default.

LogPrintf now logs using the `UNCONDITIONAL_RATE_LIMITED` category which
will start to apply rate limiting in a later commit.

For some log locations it might be safe to allow more frequent logging
without rate limiting. These locations should use the
`UNCONDITIONAL_ALWAYS` category.

[validation] Exempt UpdateTipLog from rate limiting

UpdateTipLog logs everytime a new tip is activated. This occurs at an
increased frequency during IBD and should therefore be exempt from rate
limiting.

[log] Add rate limiting to LogPrintf

To mitigate disk filling attacks caused by unsafe usages of LogPrintf,
we rate limit LogPrintf by using the fixed window rate limiter
(BCLog::LogRatelimiter) introduced in an earlier commit.

The rate limiting logic is applied per source location instead of
globally. A source location is allowed to log up to 1 MiB per hour.
Source locations that violate the limit will have their logs supressed
for up to one hour.

[test util] Mark ~DebugLogHelper as noexcept(false)

We mark ~DebugLogHelper as noexcept(false) to be able to catch the
exception it throws. This lets us use it in test in combination with
BOOST_CHECK_THROW and BOOST_CHECK_NO_THROW to check that certain log
messages are (not) logged.

[test] Check for expected log rate limiting messages

[test] Test for expected file size changes when rate limiting is enabled

[test] Check that log rate limiting is disabled for exempt source locations

[test] Check that rate limiting can be disabled
This commit is contained in:
Niklas Gögge 2021-12-29 21:15:53 +01:00 committed by Tom Trevethan
parent 817c68a64f
commit d23b6c86d0
9 changed files with 362 additions and 50 deletions

View file

@ -2715,7 +2715,7 @@ static void UpdateTipLog(
{
AssertLockHeld(::cs_main);
LogPrintf("%s%s: new best=%s height=%d version=0x%08x log2_work=%f tx=%lu date='%s' progress=%f cache=%.1fMiB(%utxo)%s\n",
LogPrint(BCLog::UNCONDITIONAL_ALWAYS,"%s%s: new best=%s height=%d version=0x%08x log2_work=%f tx=%lu date='%s' progress=%f cache=%.1fMiB(%utxo)%s\n",
prefix, func_name,
tip->GetBlockHash().ToString(), tip->nHeight, tip->nVersion,
log(tip->nChainWork.getdouble()) / log(2.0), (unsigned long)tip->nChainTx,