Updated documentation

* The mentioned Debian repository is now considered beta, make the
  wording much less scary.
* Documented how to link rocksdb dynamically and what are the pros/cons
* Documented how to cross compile
* Added `--locked` parameter to `cargo build` to prevent `cargo` from
  updating lock file
* Use `./target/release/electrs` instead of `cargo run` to avoid
  rebuilding footguns
* Added a link to in-depth explanation of cookie security
This commit is contained in:
Martin Habovstiak 2020-11-02 21:49:37 +01:00
parent 7bcd20cd56
commit d98f746404

View file

@ -4,6 +4,8 @@
### Build dependencies
Note for Raspberry Pi 4 owners: the old versions of OS/toolchains produce broken binaries. Make sure to use latest OS! (see #226)
Install [recent Rust](https://rustup.rs/) (1.41.1+, `apt install cargo` is preferred for Debian 10),
[latest Bitcoin Core](https://bitcoincore.org/en/download/) (0.16+)
and [latest Electrum wallet](https://electrum.org/#download) (3.3+).
@ -14,23 +16,112 @@ $ sudo apt update
$ sudo apt install clang cmake build-essential # for building 'rust-rocksdb'
```
Note for Raspberry Pi 4 owners: the old versions of OS/toolchains produce broken binaries. Make sure to use latest OS! (see #226)
There are two ways to compile `electrs`: by statically linking to `librocksdb` or dynamically linking.
The advantages of static linking:
* The binary is self-contained and doesn't need other dependencies, it can be transferred to other machine without worryingg
* The binary should work pretty much with every common distro
* Different library installed elsewhere doesn't affect the behavior of `electrs`
The advantages of dynamic linking:
* If a (security) bug is found in the library, you only need to upgrade/recompile the library to fix it, no need to recompile `electrs`
* Updating rocksdb can be as simple as `apt upgrade`
* The build is significantly faster (if you already have the binary version of the library from packages)
* The build is deterministic
* Cross compilation is more reliable
* If another application is also using `rocksdb`, you don't store it on disk and in RAM twice
If you decided to use dynamic linking, you will also need to install the library. On Debian:
```bash
$ sudo apt install librocksdb-dev
```
#### Preparing for cross compilation
Cross compilation can save you some time since you can compile `electrs` for a slower computer (like Raspberry Pi) on a faster machine
even with different CPU architecture. Skip this if it's not your case.
If you want to cross-compile, you need to install some additional packages.
These cross compilation instructions use `aarch64`/`arm64` + Linux as an example. (The resulting binary should work on RPi 4 with aarch64-enabled OS).
Change to your desired architecture/OS.
If you use Debian (or a derived distribution) you need to enable the target architecture:
```
$ sudo dpkg --add-architecture arm64
$ sudo apt update
```
If you use `cargo` from the repository
```bash
$ sudo apt install gcc-aarch64-linux-gnu gcc-aarch64-linux-gnu libc6-dev:arm64 libstd-rust-dev:arm64
```
If you use Rustup:
```bash
$ sudo apt install gcc-aarch64-linux-gnu gcc-aarch64-linux-gnu libc6-dev:arm64
$ rustup target add aarch64-unknown-linux-gnu
```
If you decided to use the system rocksdb (recommended if the target OS supports it), you need the version from the other architecture:
```bash
$ sudo apt install librocksdb-dev:arm64
```
#### Preparing man page generation (optional)
Optionally, you may install [`cfg_me`](https://github.com/Kixunil/cfg_me) tool for generating the manual page. The easiest way is to run `cargo install cfg_me`.
### Build
#### Download electrs
First build should take ~20 minutes:
```bash
$ git clone https://github.com/romanz/electrs
$ cd electrs
$ cargo build --release
```
### Build
#### Static linking
First build should take ~20 minutes:
```bash
$ cargo build --locked --release
```
#### Dynamic linking
```
$ ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib cargo build --locked --no-default-features --release
```
(Don't worry about `--no-default-features`, it's only related to rocksdb linking.)
#### Cross compilation
Run one of the commands above (depending on linking type) with argument `--target aarch64-unknown-linux-gnu` and prepended with env vars: `BINDGEN_EXTRA_CLANG_ARGS="-target gcc-aarch64-linux-gnu" RUSTFLAGS="-C linker=aarch64-linux-gnu-gcc"`
E.g. for dynamic linking case:
```
$ ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib BINDGEN_EXTRA_CLANG_ARGS="-target gcc-aarch64-linux-gnu" RUSTFLAGS="-C linker=aarch64-linux-gnu-gcc" cargo build --locked --release --target aarch64-unknown-linux-gnu
```
It's a bit long but sufficient! You will find the resulting binary in `target/aarch64-unknown-linux-gnu/release/electrs` - copy it to your target machine.
#### Generating man pages
If you installed `cfg_me` to generate man page, you can run `cfg_me man` to see it right away or `cfg_me -o electrs.1 man` to save it into a file (`electrs.1`).
## Docker-based installation from source
Note: currently Docker installation links statically
```bash
$ docker build -t electrs-app .
$ docker run --network host \
@ -42,18 +133,20 @@ $ docker run --network host \
## Native OS packages
There are currently no official/stable binary pckages.
There are currently no official/stable binary packages.
However, there's an [**experimental** repository for Debian 10](https://deb.ln-ask.me) (should work on recent Ubuntu, but not tested well-enough). The repository provides several significant advantages:
However, there's a [*beta* repository for Debian 10](https://deb.ln-ask.me) (should work on recent Ubuntu, but not tested well-enough). The repository provides several significant advantages:
* Everything is completely automatic - after installing `electrs` via `apt`, it's running and will automatically run on reboot, restart after crash... It also connects to bitcoind out-of-the-box, no messing with config files or anything else. It just works.
* Prebuilt binaries save you a lot of time. The binary installation of all the components is under 3 minutes on common hardware. Building from source is much longer.
* The repository contains some seurity hardening out-of-the-box - separate users for services, use of [btc-rpc-proxy](https://github.com/Kixunil/btc-rpc-proxy), etc.
And two significant disadvantages:
And two disadvantages:
* It's currently impossible to independently verify the built packages, so you have to trust the author of the repository. This will hopefully change in the future.
* The repository is considered experimental and not well tested yet. The author of the repository is also a contributor to `electrs` and appreciates [bug reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues), [test reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues/61), and other contributions.
* It's currently not trivial to independently verify the built packages, so you may need to trust the author of the repository.
The build is now deterministic but nobody verified it independently yet.
* The repository is considered beta. `electrs` seems to work well so far but was not tested heavily.
The author of the repository is also a contributor to `electrs` and appreciates [bug reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues), [test reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues/61), and other contributions.
## Manual configuration
@ -63,7 +156,7 @@ This applies only if you do **not** use some other automated systems such as Deb
Pruning must be turned **off** for `electrs` to work. `txindex` is allowed but unnecessary for `electrs`. However, you might still need it if you run other services (e.g.`eclair`)
The highly recommended way of authenticating `electrs` is using cookie file. It's the most secure and robust method. Set `rpccookiefile` option of `bitcoind` to a file within an existing directory which it can access. You can skip it if you're running both daemons under the same user and with the default directories.
The highly recommended way of authenticating `electrs` is using cookie file. It's the most [secure](https://github.com/Kixunil/security_writings/blob/master/cookie_files.md) and robust method. Set `rpccookiefile` option of `bitcoind` to a file within an existing directory which it can access. You can skip it if you're running both daemons under the same user and with the default directories.
`electrs` will wait for `bitcoind` to sync, however, you will be unabe to use it until the syncing is done.
@ -93,7 +186,7 @@ Otherwise, [`~/.bitcoin/.cookie`](https://github.com/bitcoin/bitcoin/blob/021218
First index sync should take ~1.5 hours (on a dual core Intel CPU @ 3.3 GHz, 8 GB RAM, 1TB WD Blue HDD):
```bash
$ cargo run --release -- -vvv --timestamp --db-dir ./db --electrum-rpc-addr="127.0.0.1:50001"
$ ./target/release/electrs -vvv --timestamp --db-dir ./db --electrum-rpc-addr="127.0.0.1:50001"
2018-08-17T18:27:42 - INFO - NetworkInfo { version: 179900, subversion: "/Satoshi:0.17.99/" }
2018-08-17T18:27:42 - INFO - BlockchainInfo { chain: "main", blocks: 537204, headers: 537204, bestblockhash: "0000000000000000002956768ca9421a8ddf4e53b1d81e429bd0125a383e3636", pruned: false, initialblockdownload: false }
2018-08-17T18:27:42 - DEBUG - opening DB at "./db/mainnet"
@ -123,7 +216,7 @@ Note that the final DB size should be ~20% of the `blk*.dat` files, but it may i
If initial sync fails due to `memory allocation of xxxxxxxx bytes failedAborted` errors, as may happen on devices with limited RAM, try the following arguments when starting `electrs`. It should take roughly 18 hours to sync and compact the index on an ODROID-HC1 with 8 CPU cores @ 2GHz, 2GB RAM, and an SSD using the following command:
```bash
$ cargo run --release -- -vvvv --index-batch-size=10 --jsonrpc-import --db-dir ./db --electrum-rpc-addr="127.0.0.1:50001"
$ ./target/release/electrs -vvvv --index-batch-size=10 --jsonrpc-import --db-dir ./db --electrum-rpc-addr="127.0.0.1:50001"
```
The index database is stored here:
@ -136,7 +229,7 @@ See below for [extra configuration suggestions](https://github.com/romanz/electr
## Electrum client
If you happen to use the Electrum client from [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it's pre-configured out-of-the-box already. Read below otherwise.
If you happen to use the Electrum client from [the *beta* Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it's pre-configured out-of-the-box already. Read below otherwise.
There's a prepared script for launching `electrum` in such way to connect only to the local `electrs` instance to protect your privacy.
@ -211,7 +304,7 @@ HiddenServiceVersion 3
HiddenServicePort 50001 127.0.0.1:50001
```
If you use [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it is cleaner to install `tor-hs-patch-config` using `apt` and then placing the configuration into a file inside `/etc/tor/hidden-services.d`.
If you use [the *beta* Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it is cleaner to install `tor-hs-patch-config` using `apt` and then placing the configuration into a file inside `/etc/tor/hidden-services.d`.
Restart the service:
```
@ -233,7 +326,7 @@ For more details, see http://docs.electrum.org/en/latest/tor.html.
### Sample Systemd Unit File
If you use [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), you should skip this section, as the appropriate systemd unit file is installed automatically.
If you use [the *beta* Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), you should skip this section, as the appropriate systemd unit file is installed automatically.
You may wish to have systemd manage electrs so that it's "always on." Here is a sample unit file (which assumes that the bitcoind unit file is `bitcoind.service`):