elements/src/rest.cpp
MarcoFalke 27cfaeed1e
Merge bitcoin/bitcoin#24098: rest: Use query parameters to control resource loading
54b39cfb34 Add release notes (stickies-v)
f959fc0397 Update /<count>/ endpoints to use a '?count=' query parameter instead (stickies-v)
a09497614e Add GetQueryParameter helper function (stickies-v)
fff771ee86 Handle query string when parsing data format (stickies-v)
c1aad1b3b9 scripted-diff: rename RetFormat to RESTResponseFormat (stickies-v)
9f1c54787c Refactoring: move declarations to rest.h (stickies-v)

Pull request description:

  In RESTful APIs, [typically](https://rapidapi.com/blog/api-glossary/parameters/query/) path parameters  (e.g. `/some/unique/resource/`) are used to represent resources, and query parameters (e.g. `?sort=asc`) are used to control how these resources are being loaded through e.g. sorting, pagination, filtering, ...

  As first [discussed in #17631](https://github.com/bitcoin/bitcoin/pull/17631#discussion_r733031180), the [current REST api](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) contains two endpoints `/headers/` and `/blockfilterheaders/` that rather unexpectedly use path parameters to control how many (filter) headers are returned in the response. While this is no critical issue, it is unintuitive and we are still early enough to easily phase this behaviour out and ensure new endpoints (if any) do not have to stick to non-standard behaviour just for internal consistency.

  In this PR, a new `HTTPRequest::GetQueryParameter` method is introduced to easily parse query parameters, as well as two new `/headers/` and `/blockfilterheaders/` endpoints that use a count query parameter are introduced. The old path parameter-based endpoints are kept without too much overhead, but the documentation now points to the new query parameter-based endpoints as the default interface to encourage standardness.

  ## Behaviour change
  ### New endpoints and default values
  `/headers/` and `/blockfilterheaders/` now have 2 new endpoints that contain query parameters (`?count=<count>`) instead of path parameters (`/<count>/`), as described in REST-interface.md. Since query parameters can easily have default values, I have set this at 5 for both endpoints.

  **headers**
  `GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  **blockfilterheaders**
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  ### Some previously invalid API calls are now valid
  API calls that contained query strings in the URI could not be parsed prior to this PR. This PR changes behaviour in that previously invalid calls (e.g. `GET /rest/headers/5/somehash.json?someunusedparam=foo`) would now become valid, as the query parameters are properly parsed, and discarded if unused.
  For example, prior to this PR, adding an irrelevant `someparam` parameter would be illegal:
  ```
  GET /rest/headers/5/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ->
  Invalid hash: 0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ```
  **This behaviour change affects all rest endpoints, not just the 2 new ones introduced here.**

  *(Note: I'd be open to implementing additional logic to refuse requests containing unrecognized query parameters to minimize behaviour change, but for the endpoints that we currently have I don't really see the point for that added complexity. E.g. I don't see any scenarios where misspelling a parameter could lead to harmful outcomes)*

  ## Using the REST API

  To run the API HTTP server, start a bitcoind instance with the `-rest` flag enabled. To use the
  `blockfilterheaders` endpoint, you'll also need to set `-blockfilterindex=1`:
  ```
  ./bitcoind -signet -rest -blockfilterindex=1
  ```

  As soon as bitcoind is fully up and running, you should be able to query the API, for example by
  using curl on the command line: ```curl "127.0.0.1:38332/rest/chaininfo.json"```.
  To more easily parse the JSON output, you can also use tools like 'jq' or `json_pp`, e.g.:
  ```
  curl -s "localhost:38332/rest/blockfilterheaders/basic/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?count=2" | json_pp .
  ```

  ## To do
  - [x] update `doc/release-notes`

  ## Feedback
  This is my first PR (hooray!). Please don't hold back on any feedback/comments/nits/... you may have, big or small, whether they are code, process, language, ... related. I welcome private messages too if there's anything you don't want to clutter the PR with. I'm here to learn and am grateful for everyone's input.

ACKs for top commit:
  stickies-v:
    I've had to push a tiny doc update to `REST-interface.md` (`git range-diff 219d728 9aac438 54b39cf`) since this was not merged for v23, but since there are no significant changes beyond theStack and jnewbery's ACKs I think this PR is now ready to be considered for merging? @MarcoFalke
  jnewbery:
    ACK 54b39cfb34
  theStack:
    re-ACK 54b39cfb34

Tree-SHA512: 3b393ffde34f25605ca12c0b1300799a19684b816a1d03aed38b0f5439df47bfe6a589ffbcd7b83fd2def6c9d00a1bae5e45b1d18df4ae998c617c709990f83f
2022-04-06 09:25:56 +02:00

979 lines
34 KiB
C++

// Copyright (c) 2009-2010 Satoshi Nakamoto
// Copyright (c) 2009-2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <rest.h>
#include <blockfilter.h>
#include <chain.h>
#include <chainparams.h>
#include <core_io.h>
#include <httpserver.h>
#include <index/blockfilterindex.h>
#include <index/txindex.h>
#include <node/blockstorage.h>
#include <node/context.h>
#include <primitives/block.h>
#include <primitives/transaction.h>
#include <rpc/blockchain.h>
#include <rpc/mempool.h>
#include <rpc/protocol.h>
#include <rpc/server.h>
#include <rpc/server_util.h>
#include <streams.h>
#include <sync.h>
#include <txmempool.h>
#include <util/check.h>
#include <util/system.h>
#include <validation.h>
#include <version.h>
#include <any>
#include <string>
#include <boost/algorithm/string.hpp>
#include <univalue.h>
using node::GetTransaction;
using node::IsBlockPruned;
using node::NodeContext;
using node::ReadBlockFromDisk;
static const size_t MAX_GETUTXOS_OUTPOINTS = 15; //allow a max of 15 outpoints to be queried at once
static constexpr unsigned int MAX_REST_HEADERS_RESULTS = 2000;
static const struct {
RESTResponseFormat rf;
const char* name;
} rf_names[] = {
{RESTResponseFormat::UNDEF, ""},
{RESTResponseFormat::BINARY, "bin"},
{RESTResponseFormat::HEX, "hex"},
{RESTResponseFormat::JSON, "json"},
};
struct CCoin {
uint32_t nHeight;
CTxOut out;
CCoin() : nHeight(0) {}
explicit CCoin(Coin&& in) : nHeight(in.nHeight), out(std::move(in.out)) {}
SERIALIZE_METHODS(CCoin, obj)
{
uint32_t nTxVerDummy = 0;
READWRITE(nTxVerDummy, obj.nHeight, obj.out);
}
};
static bool RESTERR(HTTPRequest* req, enum HTTPStatusCode status, std::string message)
{
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(status, message + "\r\n");
return false;
}
/**
* Get the node context.
*
* @param[in] req The HTTP request, whose status code will be set if node
* context is not found.
* @returns Pointer to the node context or nullptr if not found.
*/
static NodeContext* GetNodeContext(const std::any& context, HTTPRequest* req)
{
auto node_context = util::AnyPtr<NodeContext>(context);
if (!node_context) {
RESTERR(req, HTTP_INTERNAL_SERVER_ERROR,
strprintf("%s:%d (%s)\n"
"Internal bug detected: Node context not found!\n"
"You may report this issue here: %s\n",
__FILE__, __LINE__, __func__, PACKAGE_BUGREPORT));
return nullptr;
}
return node_context;
}
/**
* Get the node context mempool.
*
* @param[in] req The HTTP request, whose status code will be set if node
* context mempool is not found.
* @returns Pointer to the mempool or nullptr if no mempool found.
*/
static CTxMemPool* GetMemPool(const std::any& context, HTTPRequest* req)
{
auto node_context = util::AnyPtr<NodeContext>(context);
if (!node_context || !node_context->mempool) {
RESTERR(req, HTTP_NOT_FOUND, "Mempool disabled or instance not found");
return nullptr;
}
return node_context->mempool.get();
}
/**
* Get the node context chainstatemanager.
*
* @param[in] req The HTTP request, whose status code will be set if node
* context chainstatemanager is not found.
* @returns Pointer to the chainstatemanager or nullptr if none found.
*/
static ChainstateManager* GetChainman(const std::any& context, HTTPRequest* req)
{
auto node_context = util::AnyPtr<NodeContext>(context);
if (!node_context || !node_context->chainman) {
RESTERR(req, HTTP_INTERNAL_SERVER_ERROR,
strprintf("%s:%d (%s)\n"
"Internal bug detected: Chainman disabled or instance not found!\n"
"You may report this issue here: %s\n",
__FILE__, __LINE__, __func__, PACKAGE_BUGREPORT));
return nullptr;
}
return node_context->chainman.get();
}
RESTResponseFormat ParseDataFormat(std::string& param, const std::string& strReq)
{
// Remove query string (if any, separated with '?') as it should not interfere with
// parsing param and data format
param = strReq.substr(0, strReq.rfind('?'));
const std::string::size_type pos_format{param.rfind('.')};
// No format string is found
if (pos_format == std::string::npos) {
return rf_names[0].rf;
}
// Match format string to available formats
const std::string suffix(param, pos_format + 1);
for (const auto& rf_name : rf_names) {
if (suffix == rf_name.name) {
param.erase(pos_format);
return rf_name.rf;
}
}
// If no suffix is found, return RESTResponseFormat::UNDEF and original string without query string
return rf_names[0].rf;
}
static std::string AvailableDataFormatsString()
{
std::string formats;
for (const auto& rf_name : rf_names) {
if (strlen(rf_name.name) > 0) {
formats.append(".");
formats.append(rf_name.name);
formats.append(", ");
}
}
if (formats.length() > 0)
return formats.substr(0, formats.length() - 2);
return formats;
}
static bool CheckWarmup(HTTPRequest* req)
{
std::string statusmessage;
if (RPCIsInWarmup(&statusmessage))
return RESTERR(req, HTTP_SERVICE_UNAVAILABLE, "Service temporarily unavailable: " + statusmessage);
return true;
}
static bool rest_headers(const std::any& context,
HTTPRequest* req,
const std::string& strURIPart)
{
if (!CheckWarmup(req))
return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
std::vector<std::string> path;
boost::split(path, param, boost::is_any_of("/"));
std::string raw_count;
std::string hashStr;
if (path.size() == 2) {
// deprecated path: /rest/headers/<count>/<hash>
hashStr = path[1];
raw_count = path[0];
} else if (path.size() == 1) {
// new path with query parameter: /rest/headers/<hash>?count=<count>
hashStr = path[0];
raw_count = req->GetQueryParameter("count").value_or("5");
} else {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid URI format. Expected /rest/headers/<hash>.<ext>?count=<count>");
}
const auto parsed_count{ToIntegral<size_t>(raw_count)};
if (!parsed_count.has_value() || *parsed_count < 1 || *parsed_count > MAX_REST_HEADERS_RESULTS) {
return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Header count is invalid or out of acceptable range (1-%u): %s", MAX_REST_HEADERS_RESULTS, raw_count));
}
uint256 hash;
if (!ParseHashStr(hashStr, hash))
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid hash: " + hashStr);
const CBlockIndex* tip = nullptr;
std::vector<const CBlockIndex*> headers;
headers.reserve(*parsed_count);
{
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
LOCK(cs_main);
CChain& active_chain = chainman.ActiveChain();
tip = active_chain.Tip();
const CBlockIndex* pindex = chainman.m_blockman.LookupBlockIndex(hash);
while (pindex != nullptr && active_chain.Contains(pindex)) {
headers.push_back(pindex);
if (headers.size() == *parsed_count) {
break;
}
pindex = active_chain.Next(pindex);
}
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ssHeader(SER_NETWORK, PROTOCOL_VERSION);
for (const CBlockIndex *pindex : headers) {
ssHeader << pindex->GetBlockHeader();
}
std::string binaryHeader = ssHeader.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, binaryHeader);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssHeader(SER_NETWORK, PROTOCOL_VERSION);
for (const CBlockIndex *pindex : headers) {
ssHeader << pindex->GetBlockHeader();
}
std::string strHex = HexStr(ssHeader) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue jsonHeaders(UniValue::VARR);
for (const CBlockIndex *pindex : headers) {
jsonHeaders.push_back(blockheaderToJSON(tip, pindex));
}
std::string strJSON = jsonHeaders.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static bool rest_block(const std::any& context,
HTTPRequest* req,
const std::string& strURIPart,
TxVerbosity tx_verbosity)
{
if (!CheckWarmup(req))
return false;
std::string hashStr;
const RESTResponseFormat rf = ParseDataFormat(hashStr, strURIPart);
uint256 hash;
if (!ParseHashStr(hashStr, hash))
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid hash: " + hashStr);
CBlock block;
const CBlockIndex* pblockindex = nullptr;
const CBlockIndex* tip = nullptr;
{
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
LOCK(cs_main);
tip = chainman.ActiveChain().Tip();
pblockindex = chainman.m_blockman.LookupBlockIndex(hash);
if (!pblockindex) {
return RESTERR(req, HTTP_NOT_FOUND, hashStr + " not found");
}
if (IsBlockPruned(pblockindex))
return RESTERR(req, HTTP_NOT_FOUND, hashStr + " not available (pruned data)");
if (!ReadBlockFromDisk(block, pblockindex, Params().GetConsensus()))
return RESTERR(req, HTTP_NOT_FOUND, hashStr + " not found");
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ssBlock(SER_NETWORK, PROTOCOL_VERSION | RPCSerializationFlags());
ssBlock << block;
std::string binaryBlock = ssBlock.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, binaryBlock);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssBlock(SER_NETWORK, PROTOCOL_VERSION | RPCSerializationFlags());
ssBlock << block;
std::string strHex = HexStr(ssBlock) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue objBlock = blockToJSON(block, tip, pblockindex, tx_verbosity);
std::string strJSON = objBlock.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static bool rest_block_extended(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
return rest_block(context, req, strURIPart, TxVerbosity::SHOW_DETAILS_AND_PREVOUT);
}
static bool rest_block_notxdetails(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
return rest_block(context, req, strURIPart, TxVerbosity::SHOW_TXID);
}
static bool rest_filter_header(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req)) return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
std::vector<std::string> uri_parts;
boost::split(uri_parts, param, boost::is_any_of("/"));
std::string raw_count;
std::string raw_blockhash;
if (uri_parts.size() == 3) {
// deprecated path: /rest/blockfilterheaders/<filtertype>/<count>/<blockhash>
raw_blockhash = uri_parts[2];
raw_count = uri_parts[1];
} else if (uri_parts.size() == 2) {
// new path with query parameter: /rest/blockfilterheaders/<filtertype>/<blockhash>?count=<count>
raw_blockhash = uri_parts[1];
raw_count = req->GetQueryParameter("count").value_or("5");
} else {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid URI format. Expected /rest/blockfilterheaders/<filtertype>/<blockhash>.<ext>?count=<count>");
}
const auto parsed_count{ToIntegral<size_t>(raw_count)};
if (!parsed_count.has_value() || *parsed_count < 1 || *parsed_count > MAX_REST_HEADERS_RESULTS) {
return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Header count is invalid or out of acceptable range (1-%u): %s", MAX_REST_HEADERS_RESULTS, raw_count));
}
uint256 block_hash;
if (!ParseHashStr(raw_blockhash, block_hash)) {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid hash: " + raw_blockhash);
}
BlockFilterType filtertype;
if (!BlockFilterTypeByName(uri_parts[0], filtertype)) {
return RESTERR(req, HTTP_BAD_REQUEST, "Unknown filtertype " + uri_parts[0]);
}
BlockFilterIndex* index = GetBlockFilterIndex(filtertype);
if (!index) {
return RESTERR(req, HTTP_BAD_REQUEST, "Index is not enabled for filtertype " + uri_parts[0]);
}
std::vector<const CBlockIndex*> headers;
headers.reserve(*parsed_count);
{
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
LOCK(cs_main);
CChain& active_chain = chainman.ActiveChain();
const CBlockIndex* pindex = chainman.m_blockman.LookupBlockIndex(block_hash);
while (pindex != nullptr && active_chain.Contains(pindex)) {
headers.push_back(pindex);
if (headers.size() == *parsed_count)
break;
pindex = active_chain.Next(pindex);
}
}
bool index_ready = index->BlockUntilSyncedToCurrentChain();
std::vector<uint256> filter_headers;
filter_headers.reserve(*parsed_count);
for (const CBlockIndex* pindex : headers) {
uint256 filter_header;
if (!index->LookupFilterHeader(pindex, filter_header)) {
std::string errmsg = "Filter not found.";
if (!index_ready) {
errmsg += " Block filters are still in the process of being indexed.";
} else {
errmsg += " This error is unexpected and indicates index corruption.";
}
return RESTERR(req, HTTP_NOT_FOUND, errmsg);
}
filter_headers.push_back(filter_header);
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ssHeader{SER_NETWORK, PROTOCOL_VERSION};
for (const uint256& header : filter_headers) {
ssHeader << header;
}
std::string binaryHeader = ssHeader.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, binaryHeader);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssHeader{SER_NETWORK, PROTOCOL_VERSION};
for (const uint256& header : filter_headers) {
ssHeader << header;
}
std::string strHex = HexStr(ssHeader) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue jsonHeaders(UniValue::VARR);
for (const uint256& header : filter_headers) {
jsonHeaders.push_back(header.GetHex());
}
std::string strJSON = jsonHeaders.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static bool rest_block_filter(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req)) return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
// request is sent over URI scheme /rest/blockfilter/filtertype/blockhash
std::vector<std::string> uri_parts;
boost::split(uri_parts, param, boost::is_any_of("/"));
if (uri_parts.size() != 2) {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid URI format. Expected /rest/blockfilter/<filtertype>/<blockhash>");
}
uint256 block_hash;
if (!ParseHashStr(uri_parts[1], block_hash)) {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid hash: " + uri_parts[1]);
}
BlockFilterType filtertype;
if (!BlockFilterTypeByName(uri_parts[0], filtertype)) {
return RESTERR(req, HTTP_BAD_REQUEST, "Unknown filtertype " + uri_parts[0]);
}
BlockFilterIndex* index = GetBlockFilterIndex(filtertype);
if (!index) {
return RESTERR(req, HTTP_BAD_REQUEST, "Index is not enabled for filtertype " + uri_parts[0]);
}
const CBlockIndex* block_index;
bool block_was_connected;
{
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
LOCK(cs_main);
block_index = chainman.m_blockman.LookupBlockIndex(block_hash);
if (!block_index) {
return RESTERR(req, HTTP_NOT_FOUND, uri_parts[1] + " not found");
}
block_was_connected = block_index->IsValid(BLOCK_VALID_SCRIPTS);
}
bool index_ready = index->BlockUntilSyncedToCurrentChain();
BlockFilter filter;
if (!index->LookupFilter(block_index, filter)) {
std::string errmsg = "Filter not found.";
if (!block_was_connected) {
errmsg += " Block was not connected to active chain.";
} else if (!index_ready) {
errmsg += " Block filters are still in the process of being indexed.";
} else {
errmsg += " This error is unexpected and indicates index corruption.";
}
return RESTERR(req, HTTP_NOT_FOUND, errmsg);
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ssResp{SER_NETWORK, PROTOCOL_VERSION};
ssResp << filter;
std::string binaryResp = ssResp.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, binaryResp);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssResp{SER_NETWORK, PROTOCOL_VERSION};
ssResp << filter;
std::string strHex = HexStr(ssResp) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue ret(UniValue::VOBJ);
ret.pushKV("filter", HexStr(filter.GetEncodedFilter()));
std::string strJSON = ret.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
// A bit of a hack - dependency on a function defined in rpc/blockchain.cpp
RPCHelpMan getblockchaininfo();
static bool rest_chaininfo(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req))
return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
switch (rf) {
case RESTResponseFormat::JSON: {
JSONRPCRequest jsonRequest;
jsonRequest.context = context;
jsonRequest.params = UniValue(UniValue::VARR);
UniValue chainInfoObject = getblockchaininfo().HandleRequest(jsonRequest);
std::string strJSON = chainInfoObject.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: json)");
}
}
}
static bool rest_mempool_info(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req))
return false;
const CTxMemPool* mempool = GetMemPool(context, req);
if (!mempool) return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
switch (rf) {
case RESTResponseFormat::JSON: {
UniValue mempoolInfoObject = MempoolInfoToJSON(*mempool);
std::string strJSON = mempoolInfoObject.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: json)");
}
}
}
static bool rest_mempool_contents(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req)) return false;
const CTxMemPool* mempool = GetMemPool(context, req);
if (!mempool) return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
switch (rf) {
case RESTResponseFormat::JSON: {
UniValue mempoolObject = MempoolToJSON(*mempool, true);
std::string strJSON = mempoolObject.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: json)");
}
}
}
static bool rest_tx(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req))
return false;
std::string hashStr;
const RESTResponseFormat rf = ParseDataFormat(hashStr, strURIPart);
uint256 hash;
if (!ParseHashStr(hashStr, hash))
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid hash: " + hashStr);
if (g_txindex) {
g_txindex->BlockUntilSyncedToCurrentChain();
}
const NodeContext* const node = GetNodeContext(context, req);
if (!node) return false;
uint256 hashBlock = uint256();
const CTransactionRef tx = GetTransaction(/* block_index */ nullptr, node->mempool.get(), hash, Params().GetConsensus(), hashBlock);
if (!tx) {
return RESTERR(req, HTTP_NOT_FOUND, hashStr + " not found");
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION | RPCSerializationFlags());
ssTx << tx;
std::string binaryTx = ssTx.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, binaryTx);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION | RPCSerializationFlags());
ssTx << tx;
std::string strHex = HexStr(ssTx) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue objTx(UniValue::VOBJ);
TxToUniv(*tx, /*block_hash=*/hashBlock, /*entry=*/ objTx);
std::string strJSON = objTx.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static bool rest_getutxos(const std::any& context, HTTPRequest* req, const std::string& strURIPart)
{
if (!CheckWarmup(req))
return false;
std::string param;
const RESTResponseFormat rf = ParseDataFormat(param, strURIPart);
std::vector<std::string> uriParts;
if (param.length() > 1)
{
std::string strUriParams = param.substr(1);
boost::split(uriParts, strUriParams, boost::is_any_of("/"));
}
// throw exception in case of an empty request
std::string strRequestMutable = req->ReadBody();
if (strRequestMutable.length() == 0 && uriParts.size() == 0)
return RESTERR(req, HTTP_BAD_REQUEST, "Error: empty request");
bool fInputParsed = false;
bool fCheckMemPool = false;
std::vector<COutPoint> vOutPoints;
// parse/deserialize input
// input-format = output-format, rest/getutxos/bin requires binary input, gives binary output, ...
if (uriParts.size() > 0)
{
//inputs is sent over URI scheme (/rest/getutxos/checkmempool/txid1-n/txid2-n/...)
if (uriParts[0] == "checkmempool") fCheckMemPool = true;
for (size_t i = (fCheckMemPool) ? 1 : 0; i < uriParts.size(); i++)
{
uint256 txid;
int32_t nOutput;
std::string strTxid = uriParts[i].substr(0, uriParts[i].find('-'));
std::string strOutput = uriParts[i].substr(uriParts[i].find('-')+1);
if (!ParseInt32(strOutput, &nOutput) || !IsHex(strTxid))
return RESTERR(req, HTTP_BAD_REQUEST, "Parse error");
txid.SetHex(strTxid);
vOutPoints.push_back(COutPoint(txid, (uint32_t)nOutput));
}
if (vOutPoints.size() > 0)
fInputParsed = true;
else
return RESTERR(req, HTTP_BAD_REQUEST, "Error: empty request");
}
switch (rf) {
case RESTResponseFormat::HEX: {
// convert hex to bin, continue then with bin part
std::vector<unsigned char> strRequestV = ParseHex(strRequestMutable);
strRequestMutable.assign(strRequestV.begin(), strRequestV.end());
[[fallthrough]];
}
case RESTResponseFormat::BINARY: {
try {
//deserialize only if user sent a request
if (strRequestMutable.size() > 0)
{
if (fInputParsed) //don't allow sending input over URI and HTTP RAW DATA
return RESTERR(req, HTTP_BAD_REQUEST, "Combination of URI scheme inputs and raw post data is not allowed");
CDataStream oss(SER_NETWORK, PROTOCOL_VERSION);
oss << strRequestMutable;
oss >> fCheckMemPool;
oss >> vOutPoints;
}
} catch (const std::ios_base::failure&) {
// abort in case of unreadable binary data
return RESTERR(req, HTTP_BAD_REQUEST, "Parse error");
}
break;
}
case RESTResponseFormat::JSON: {
if (!fInputParsed)
return RESTERR(req, HTTP_BAD_REQUEST, "Error: empty request");
break;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
// limit max outpoints
if (vOutPoints.size() > MAX_GETUTXOS_OUTPOINTS)
return RESTERR(req, HTTP_BAD_REQUEST, strprintf("Error: max outpoints exceeded (max: %d, tried: %d)", MAX_GETUTXOS_OUTPOINTS, vOutPoints.size()));
// check spentness and form a bitmap (as well as a JSON capable human-readable string representation)
std::vector<unsigned char> bitmap;
std::vector<CCoin> outs;
std::string bitmapStringRepresentation;
std::vector<bool> hits;
bitmap.resize((vOutPoints.size() + 7) / 8);
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
{
auto process_utxos = [&vOutPoints, &outs, &hits](const CCoinsView& view, const CTxMemPool& mempool) {
for (const COutPoint& vOutPoint : vOutPoints) {
Coin coin;
bool hit = !mempool.isSpent(vOutPoint) && view.GetCoin(vOutPoint, coin);
hits.push_back(hit);
if (hit) outs.emplace_back(std::move(coin));
}
};
if (fCheckMemPool) {
const CTxMemPool* mempool = GetMemPool(context, req);
if (!mempool) return false;
// use db+mempool as cache backend in case user likes to query mempool
LOCK2(cs_main, mempool->cs);
CCoinsViewCache& viewChain = chainman.ActiveChainstate().CoinsTip();
CCoinsViewMemPool viewMempool(&viewChain, *mempool);
process_utxos(viewMempool, *mempool);
} else {
LOCK(cs_main); // no need to lock mempool!
process_utxos(chainman.ActiveChainstate().CoinsTip(), CTxMemPool());
}
for (size_t i = 0; i < hits.size(); ++i) {
const bool hit = hits[i];
bitmapStringRepresentation.append(hit ? "1" : "0"); // form a binary string representation (human-readable for json output)
bitmap[i / 8] |= ((uint8_t)hit) << (i % 8);
}
}
switch (rf) {
case RESTResponseFormat::BINARY: {
// serialize data
// use exact same output as mentioned in Bip64
CDataStream ssGetUTXOResponse(SER_NETWORK, PROTOCOL_VERSION);
ssGetUTXOResponse << chainman.ActiveChain().Height() << chainman.ActiveChain().Tip()->GetBlockHash() << bitmap << outs;
std::string ssGetUTXOResponseString = ssGetUTXOResponse.str();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, ssGetUTXOResponseString);
return true;
}
case RESTResponseFormat::HEX: {
CDataStream ssGetUTXOResponse(SER_NETWORK, PROTOCOL_VERSION);
ssGetUTXOResponse << chainman.ActiveChain().Height() << chainman.ActiveChain().Tip()->GetBlockHash() << bitmap << outs;
std::string strHex = HexStr(ssGetUTXOResponse) + "\n";
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, strHex);
return true;
}
case RESTResponseFormat::JSON: {
UniValue objGetUTXOResponse(UniValue::VOBJ);
// pack in some essentials
// use more or less the same output as mentioned in Bip64
objGetUTXOResponse.pushKV("chainHeight", chainman.ActiveChain().Height());
objGetUTXOResponse.pushKV("chaintipHash", chainman.ActiveChain().Tip()->GetBlockHash().GetHex());
objGetUTXOResponse.pushKV("bitmap", bitmapStringRepresentation);
UniValue utxos(UniValue::VARR);
for (const CCoin& coin : outs) {
UniValue utxo(UniValue::VOBJ);
utxo.pushKV("height", (int32_t)coin.nHeight);
utxo.pushKV("value", ValueFromAmount(coin.out.nValue));
// include the script in a json output
UniValue o(UniValue::VOBJ);
ScriptToUniv(coin.out.scriptPubKey, /*out=*/o, /*include_hex=*/true, /*include_address=*/true);
utxo.pushKV("scriptPubKey", o);
utxos.push_back(utxo);
}
objGetUTXOResponse.pushKV("utxos", utxos);
// return json string
std::string strJSON = objGetUTXOResponse.write() + "\n";
req->WriteHeader("Content-Type", "application/json");
req->WriteReply(HTTP_OK, strJSON);
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static bool rest_blockhash_by_height(const std::any& context, HTTPRequest* req,
const std::string& str_uri_part)
{
if (!CheckWarmup(req)) return false;
std::string height_str;
const RESTResponseFormat rf = ParseDataFormat(height_str, str_uri_part);
int32_t blockheight = -1; // Initialization done only to prevent valgrind false positive, see https://github.com/bitcoin/bitcoin/pull/18785
if (!ParseInt32(height_str, &blockheight) || blockheight < 0) {
return RESTERR(req, HTTP_BAD_REQUEST, "Invalid height: " + SanitizeString(height_str));
}
CBlockIndex* pblockindex = nullptr;
{
ChainstateManager* maybe_chainman = GetChainman(context, req);
if (!maybe_chainman) return false;
ChainstateManager& chainman = *maybe_chainman;
LOCK(cs_main);
const CChain& active_chain = chainman.ActiveChain();
if (blockheight > active_chain.Height()) {
return RESTERR(req, HTTP_NOT_FOUND, "Block height out of range");
}
pblockindex = active_chain[blockheight];
}
switch (rf) {
case RESTResponseFormat::BINARY: {
CDataStream ss_blockhash(SER_NETWORK, PROTOCOL_VERSION);
ss_blockhash << pblockindex->GetBlockHash();
req->WriteHeader("Content-Type", "application/octet-stream");
req->WriteReply(HTTP_OK, ss_blockhash.str());
return true;
}
case RESTResponseFormat::HEX: {
req->WriteHeader("Content-Type", "text/plain");
req->WriteReply(HTTP_OK, pblockindex->GetBlockHash().GetHex() + "\n");
return true;
}
case RESTResponseFormat::JSON: {
req->WriteHeader("Content-Type", "application/json");
UniValue resp = UniValue(UniValue::VOBJ);
resp.pushKV("blockhash", pblockindex->GetBlockHash().GetHex());
req->WriteReply(HTTP_OK, resp.write() + "\n");
return true;
}
default: {
return RESTERR(req, HTTP_NOT_FOUND, "output format not found (available: " + AvailableDataFormatsString() + ")");
}
}
}
static const struct {
const char* prefix;
bool (*handler)(const std::any& context, HTTPRequest* req, const std::string& strReq);
} uri_prefixes[] = {
{"/rest/tx/", rest_tx},
{"/rest/block/notxdetails/", rest_block_notxdetails},
{"/rest/block/", rest_block_extended},
{"/rest/blockfilter/", rest_block_filter},
{"/rest/blockfilterheaders/", rest_filter_header},
{"/rest/chaininfo", rest_chaininfo},
{"/rest/mempool/info", rest_mempool_info},
{"/rest/mempool/contents", rest_mempool_contents},
{"/rest/headers/", rest_headers},
{"/rest/getutxos", rest_getutxos},
{"/rest/blockhashbyheight/", rest_blockhash_by_height},
};
void StartREST(const std::any& context)
{
for (const auto& up : uri_prefixes) {
auto handler = [context, up](HTTPRequest* req, const std::string& prefix) { return up.handler(context, req, prefix); };
RegisterHTTPHandler(up.prefix, false, handler);
}
}
void InterruptREST()
{
}
void StopREST()
{
for (const auto& up : uri_prefixes) {
UnregisterHTTPHandler(up.prefix, false);
}
}