mirror of
https://github.com/btcsuite/btcd.git
synced 2026-08-13 12:32:51 +02:00
In this commit, we optimize the decoding for the CFCheckpkt message. The
old decode routine would do a fresh alloc for each hash to be read out.
Instead, we'll now allocate enough memory for the entire set of headers
to be decoded, then read them into that contiguous slice, and point to
members of this slice in the wire message itself.
We've also added benchmarks to show the improvement:
```
⛰ cat bench-cmp.txt
goos: darwin
goarch: arm64
pkg: github.com/btcsuite/btcd/wire
cpu: Apple M4 Max
│ bench-old.txt │ bench-new.txt │
│ sec/op │ sec/op vs base │
MsgCFCheckptDecode/headers_1000-16 14.354µ ± ∞ ¹ 6.919µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_10000-16 146.77µ ± ∞ ¹ 70.23µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_100000-16 1473.6µ ± ∞ ¹ 564.2µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptEncode/headers_1000-16 7.021µ ± ∞ ¹ 7.196µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptEncode/headers_10000-16 88.97µ ± ∞ ¹ 90.40µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptEncode/headers_100000-16 861.0µ ± ∞ ¹ 875.0µ ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecodeEmpty-16 68.28n ± ∞ ¹ 66.61n ± ∞ ¹ ~ (p=1.000 n=1) ²
geomean 37.98µ 26.98µ -28.98%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
│ bench-old.txt │ bench-new.txt │
│ B/op │ B/op vs base │
MsgCFCheckptDecode/headers_1000-16 39.36Ki ± ∞ ¹ 40.11Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_10000-16 392.6Ki ± ∞ ¹ 400.1Ki ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_100000-16 3.817Mi ± ∞ ¹ 3.820Mi ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptEncode/headers_1000-16 63.98Ki ± ∞ ¹ 63.98Ki ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptEncode/headers_10000-16 1.000Mi ± ∞ ¹ 1.000Mi ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptEncode/headers_100000-16 8.000Mi ± ∞ ¹ 8.000Mi ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptDecodeEmpty-16 112.0 ± ∞ ¹ 112.0 ± ∞ ¹ ~ (p=1.000 n=1) ³
geomean 166.5Ki 167.4Ki +0.55%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ all samples are equal
│ bench-old.txt │ bench-new.txt │
│ allocs/op │ allocs/op vs base │
MsgCFCheckptDecode/headers_1000-16 1003.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_10000-16 10003.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptDecode/headers_100000-16 100003.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~ (p=1.000 n=1) ²
MsgCFCheckptEncode/headers_1000-16 11.00 ± ∞ ¹ 11.00 ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptEncode/headers_10000-16 15.00 ± ∞ ¹ 15.00 ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptEncode/headers_100000-16 18.00 ± ∞ ¹ 18.00 ± ∞ ¹ ~ (p=1.000 n=1) ³
MsgCFCheckptDecodeEmpty-16 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=1) ³
geomean 179.3 6.268 -96.50%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
³ all samples are equal
```
Old bench:
```
goos: darwin
goarch: arm64
pkg: github.com/btcsuite/btcd/wire
cpu: Apple M4 Max
BenchmarkMsgCFCheckptDecode/headers_1000-16 74678 14354 ns/op 40304 B/op 1003 allocs/op
BenchmarkMsgCFCheckptDecode/headers_10000-16 8234 146770 ns/op 402033 B/op 10003 allocs/op
BenchmarkMsgCFCheckptDecode/headers_100000-16 822 1473622 ns/op 4002931 B/op 100003 allocs/op
BenchmarkMsgCFCheckptEncode/headers_1000-16 173762 7021 ns/op 65520 B/op 11 allocs/op
BenchmarkMsgCFCheckptEncode/headers_10000-16 13459 88968 ns/op 1048564 B/op 15 allocs/op
BenchmarkMsgCFCheckptEncode/headers_100000-16 1399 860985 ns/op 8388592 B/op 18 allocs/op
BenchmarkMsgCFCheckptDecodeEmpty-16 17459148 68.28 ns/op 112 B/op 2 allocs/op
PASS
ok github.com/btcsuite/btcd/wire 10.135s
```
New bench:
```
goos: darwin
goarch: arm64
pkg: github.com/btcsuite/btcd/wire
cpu: Apple M4 Max
BenchmarkMsgCFCheckptDecode/headers_1000-16 166368 6919 ns/op 41072 B/op 4 allocs/op
BenchmarkMsgCFCheckptDecode/headers_10000-16 17079 70227 ns/op 409712 B/op 4 allocs/op
BenchmarkMsgCFCheckptDecode/headers_100000-16 2062 564175 ns/op 4006003 B/op 4 allocs/op
BenchmarkMsgCFCheckptEncode/headers_1000-16 173940 7196 ns/op 65520 B/op 11 allocs/op
BenchmarkMsgCFCheckptEncode/headers_10000-16 13054 90401 ns/op 1048564 B/op 15 allocs/op
BenchmarkMsgCFCheckptEncode/headers_100000-16 1408 875012 ns/op 8388592 B/op 18 allocs/op
BenchmarkMsgCFCheckptDecodeEmpty-16 17256627 66.61 ns/op 112 B/op 2 allocs/op
PASS
ok github.com/btcsuite/btcd/wire 10.522s
```
As seen from the benchmarks, allocs have decreased by 96%, and the decode
speed by nearly 30%.
187 lines
5.9 KiB
Go
187 lines
5.9 KiB
Go
// Copyright (c) 2018 The btcsuite developers
|
|
// Use of this source code is governed by an ISC
|
|
// license that can be found in the LICENSE file.
|
|
|
|
package wire
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
|
|
"github.com/btcsuite/btcd/chaincfg/chainhash"
|
|
)
|
|
|
|
const (
|
|
// CFCheckptInterval is the gap (in number of blocks) between each
|
|
// filter header checkpoint.
|
|
CFCheckptInterval = 1000
|
|
|
|
// maxCFHeadersLen is the max number of filter headers we will attempt
|
|
// to decode.
|
|
maxCFHeadersLen = 100000
|
|
|
|
// maxCFCheckptPayload calculates the maximum reasonable payload size
|
|
// for CF checkpoint messages.
|
|
//
|
|
// Calculation: 1 byte (filter type) + 32 bytes (stop hash) +
|
|
// 5 bytes (max varint) + (maxCFHeadersLen * 32 bytes per hash)
|
|
maxCFCheckptPayload = 1 + 32 + 5 + (maxCFHeadersLen * 32)
|
|
)
|
|
|
|
// ErrInsaneCFHeaderCount signals that we were asked to decode an
|
|
// unreasonable number of cfilter headers.
|
|
var ErrInsaneCFHeaderCount = errors.New(
|
|
"refusing to decode unreasonable number of filter headers")
|
|
|
|
// MsgCFCheckpt implements the Message interface and represents a bitcoin
|
|
// cfcheckpt message. It is used to deliver committed filter header information
|
|
// in response to a getcfcheckpt message (MsgGetCFCheckpt). See MsgGetCFCheckpt
|
|
// for details on requesting the headers.
|
|
type MsgCFCheckpt struct {
|
|
FilterType FilterType
|
|
StopHash chainhash.Hash
|
|
FilterHeaders []*chainhash.Hash
|
|
}
|
|
|
|
// AddCFHeader adds a new committed filter header to the message.
|
|
func (msg *MsgCFCheckpt) AddCFHeader(header *chainhash.Hash) error {
|
|
if len(msg.FilterHeaders) == cap(msg.FilterHeaders) {
|
|
str := fmt.Sprintf("FilterHeaders has insufficient capacity for "+
|
|
"additional header: len = %d", len(msg.FilterHeaders))
|
|
return messageError("MsgCFCheckpt.AddCFHeader", str)
|
|
}
|
|
|
|
msg.FilterHeaders = append(msg.FilterHeaders, header)
|
|
return nil
|
|
}
|
|
|
|
// BtcDecode decodes r using the bitcoin protocol encoding into the receiver.
|
|
// This is part of the Message interface implementation.
|
|
func (msg *MsgCFCheckpt) BtcDecode(r io.Reader, pver uint32, _ MessageEncoding) error {
|
|
buf := binarySerializer.Borrow()
|
|
defer binarySerializer.Return(buf)
|
|
|
|
// Read filter type
|
|
if _, err := io.ReadFull(r, buf[:1]); err != nil {
|
|
return err
|
|
}
|
|
msg.FilterType = FilterType(buf[0])
|
|
|
|
// Read stop hash
|
|
if _, err := io.ReadFull(r, msg.StopHash[:]); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Read number of filter headers
|
|
count, err := ReadVarIntBuf(r, pver, buf)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// Refuse to decode an insane number of cfheaders.
|
|
if count > maxCFHeadersLen {
|
|
return ErrInsaneCFHeaderCount
|
|
}
|
|
|
|
if count == 0 {
|
|
msg.FilterHeaders = make([]*chainhash.Hash, 0)
|
|
return nil
|
|
}
|
|
|
|
// Optimize memory allocation by creating a single backing array for
|
|
// all hashes. This reduces GC pressure and improves cache locality.
|
|
hashes := make([]chainhash.Hash, count)
|
|
msg.FilterHeaders = make([]*chainhash.Hash, count)
|
|
|
|
// Now we'll read all the hashes directly into the backing array we've
|
|
// created above. We'll then point the underlying filter header hashes
|
|
// into this backing array.
|
|
for i := uint64(0); i < count; i++ {
|
|
if _, err := io.ReadFull(r, hashes[i][:]); err != nil {
|
|
return err
|
|
}
|
|
msg.FilterHeaders[i] = &hashes[i]
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// BtcEncode encodes the receiver to w using the bitcoin protocol encoding.
|
|
// This is part of the Message interface implementation.
|
|
func (msg *MsgCFCheckpt) BtcEncode(w io.Writer, pver uint32, _ MessageEncoding) error {
|
|
buf := binarySerializer.Borrow()
|
|
defer binarySerializer.Return(buf)
|
|
|
|
// Write filter type
|
|
buf[0] = byte(msg.FilterType)
|
|
if _, err := w.Write(buf[:1]); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Write stop hash
|
|
if _, err := w.Write(msg.StopHash[:]); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Write length of FilterHeaders slice
|
|
count := len(msg.FilterHeaders)
|
|
err := WriteVarIntBuf(w, pver, uint64(count), buf)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
for _, cfh := range msg.FilterHeaders {
|
|
_, err := w.Write(cfh[:])
|
|
if err != nil {
|
|
return err
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// Deserialize decodes a filter header from r into the receiver using a format
|
|
// that is suitable for long-term storage such as a database. This function
|
|
// differs from BtcDecode in that BtcDecode decodes from the bitcoin wire
|
|
// protocol as it was sent across the network. The wire encoding can
|
|
// technically differ depending on the protocol version and doesn't even really
|
|
// need to match the format of a stored filter header at all. As of the time
|
|
// this comment was written, the encoded filter header is the same in both
|
|
// instances, but there is a distinct difference and separating the two allows
|
|
// the API to be flexible enough to deal with changes.
|
|
func (msg *MsgCFCheckpt) Deserialize(r io.Reader) error {
|
|
// At the current time, there is no difference between the wire encoding
|
|
// and the stable long-term storage format. As a result, make use of
|
|
// BtcDecode.
|
|
return msg.BtcDecode(r, 0, BaseEncoding)
|
|
}
|
|
|
|
// Command returns the protocol command string for the message. This is part
|
|
// of the Message interface implementation.
|
|
func (msg *MsgCFCheckpt) Command() string {
|
|
return CmdCFCheckpt
|
|
}
|
|
|
|
// MaxPayloadLength returns the maximum length the payload can be for the
|
|
// receiver. This is part of the Message interface implementation.
|
|
func (msg *MsgCFCheckpt) MaxPayloadLength(pver uint32) uint32 {
|
|
// Use a more precise calculation based on the maximum number of
|
|
// filter headers we support. No no reason to read more than we'll
|
|
// process in BtcDecode.
|
|
return maxCFCheckptPayload
|
|
}
|
|
|
|
// NewMsgCFCheckpt returns a new bitcoin cfheaders message that conforms to the
|
|
// Message interface. See MsgCFCheckpt for details.
|
|
func NewMsgCFCheckpt(filterType FilterType, stopHash *chainhash.Hash,
|
|
headersCount int) *MsgCFCheckpt {
|
|
|
|
// We pre-allocate with an exact capacity when count is known to avoid
|
|
// slice growth during message construction.
|
|
return &MsgCFCheckpt{
|
|
FilterType: filterType,
|
|
StopHash: *stopHash,
|
|
FilterHeaders: make([]*chainhash.Hash, 0, headersCount),
|
|
}
|
|
}
|