No description
Find a file
Oliver Gugger a6c363708f
clientdb: add migration for existing orders
With this commit we add a migration that adds a fake order
creation timestamp to all orders that existed before this code is first
run.
2020-10-01 13:16:50 +02:00
account account: subscribe account updates on startup for StatePendingBatch 2020-09-23 11:24:49 -07:00
auctioneer multi: fix import order 2020-09-09 19:55:16 -07:00
chaninfo multi: update aperture, use lndclient, remove loop 2020-08-11 19:41:01 +02:00
clientdb clientdb: add migration for existing orders 2020-10-01 13:16:50 +02:00
cmd multi: add order and clearing rate to leases, show absolute expiry 2020-09-17 17:40:30 -07:00
event event: add new event package 2020-10-01 13:16:45 +02:00
internal/test fundingmgr: don't block on peer connection attempt 2020-09-17 13:29:31 +02:00
make make: add Makefile, .gitignore and lint config 2020-06-23 10:45:34 +02:00
order order+clientdb: add order specific events 2020-10-01 13:16:47 +02:00
poolrpc multi: add order and clearing rate to leases, show absolute expiry 2020-09-17 17:40:30 -07:00
poolscript pool: rename clmscript to poolrpc 2020-09-09 19:55:04 -07:00
terms clmrpc+auctioneer: add Terms RPC 2020-08-18 13:15:36 +02:00
.gitignore multi: rename project from llm to pool 2020-09-09 19:55:10 -07:00
.golangci.yml Makefile+lint: only limit concurrency on Travis 2020-06-24 09:58:17 +02:00
.travis.yml travis: disable coverage for private repo 2020-06-24 10:53:59 +02:00
config.go config+server: add macaroon authentication to pool's server connn 2020-09-17 08:37:21 +02:00
Dockerfile multi: rename project from llm to pool 2020-09-09 19:55:10 -07:00
fundingmgr.go fundingmgr: inspect grpc error code 2020-09-17 13:29:31 +02:00
fundingmgr_test.go fundingmgr_test: use manual peer list only, wait for connections 2020-09-25 13:57:48 +02:00
go.mod mod+macaroons: bump to lnd v0.11.1-beta.rc3 2020-09-18 09:00:11 +02:00
go.sum mod+macaroons: bump to lnd v0.11.1-beta.rc3 2020-09-18 09:00:11 +02:00
log.go multi: fix import order 2020-09-09 19:55:16 -07:00
macaroons.go mod+macaroons: bump to lnd v0.11.1-beta.rc3 2020-09-18 09:00:11 +02:00
Makefile multi: rename project from llm to pool 2020-09-09 19:55:10 -07:00
README.md README: update readme to include leases info, clarity on order pricing 2020-09-17 17:40:19 -07:00
rpcserver.go Merge pull request #87 from wpaulino/check-tor-addr-without-port 2020-09-18 12:48:33 -07:00
run.go multi: extract config validation 2020-09-12 09:08:13 +02:00
server.go server: allow poold to be used as external subserver 2020-09-17 08:37:22 +02:00
version.go build: bump version to v0.2.3-alpha 2020-09-18 12:49:37 -07:00

Lightning Pool

Lightning Pool is a non-custodial batched uniform clearing-price auction for Channel Liquidity Bonds (CLB). A CLB packages up inbound (or outbound!) channel liquidity (ability to send/receive funds) as a fixed incoming asset (earning interest over time) with a maturity date expressed in blocks. The maturity date of each of the channels is enforced by Bitcoin contracts, ensuring that the funds of the maker (the party that sold the channel) can't be swept until the maturity height. All cleared orders (purchased channels) are cleared in a single batched on-chain transaction.

The existence of an open auction to acquire/sell channel liquidity provides all participants on the network with a more stable income source in addition to routing network fees. By selling liquidity within the marketplace, individuals are able to price their channels to ensure that they're compensated for the time-value of their coins within a channel, accounting for worst-case force close CSV delays.

Pool critically allows participants on the network to exchange pricing signals to determine where liquidity in the network is most demanded. A channel opened to an area of the sub-graph that doesn't actually need that liquidity will likely remain dormant and not earn any active routing fees. Instead, if capital can be allocated within the network in an efficient manner, being placed where it's most demanded, we can better utilize the allocated capital on the network, and also allow new participants to easily identify where their capital is most needed.

Amongst several other uses cases, the Pool allows a new participant in the network to easily boostrap their ability to receive funds by paying only a percentage of the total amount of inbound funds acquired. As an example, a node could acquire 100 million satoshis (1000 units, more on that below) for 100,000 satoshis, or 0.1%. Ultimately the prices will be determined by the open market place.

A non-exhaustive list of use cases includes:

  • Bootstrapping new users with side car channels: A common question posted concerning the Lightning Network goes something like: Alice is new to Bitcoin entirely, how can she join the Lightning Network without her, herself, making any new on-chain Bitcoin transactions? Its desirable to a solution to onboarding new users on to the network which is as as simple as sending coins to a fresh address. The Pool solves this by allowing a third party Carol, to purchase a channel for Alice, which includes starting outbound liquidity.

  • Demand fueled routing node channel selection: Another common question with regards to the LN is: "where should I open my channels to , such that they'll actually be routed through"?. Pool provides a new signal for autopilot agents: a market demand signal. The node can offer up its liquidity and have it automatically be allocated where it's most demanded.

  • Bootstrapping new services to Lightning: Any new service launched on the Lightning Network will likely need to figure out how to obtain inbound channels so they can accept payments. For this Pool provides an elegant solution in that a merchant can set up a series of "introduction points" negotiated via the market place. The merchant can pay a small percentage of the total amount of liquidity allocated towards it, and also ensure that the funds will be committed for a set period of time.

  • Allowing users to instantly receive with a wallet: A common UX challenge that wallets face concerns ensuring a user can receive funds as soon as they set up a wallet. Some wallet providers have chosen to open new inbound channels to users themselves. This gives users the inbound bandwidth they need to receive, but can come at a high capital cost to the wallet provider as they need to commit funds with a 1:1 ratio. The Lightning Pool allows them to achieve some leverage in a sense, as they can pay only a percentage of the funds to be allocated to a new user. As an eaxmple, they can pay 1000 satohis to have 1 million satoshis be alloacted to a user.

The Auction Lifecycle

In this section, we'll walk through the typical auction lifecycle, and in the process explain some key components of the Pool, and also illustrate how to drive your poold on the command line.

Accounts

Like any exchange/auction, before you can start trading, you'll need an account! Accounts in the Pool are actually special on-chain contracts. A user deposits a certain amount of funds into an account which has a set expiry. By having users commit funds to an account in order to place orders, we ensure that they're unable to spoof orders (placing an order that they can't fulfill). We also add a cost to attempting to sybil attack the venue as well.

The script for an account is very simple, funds can be moved from the account:

  • With a joint 2-of-2 signature by the auctioneer and the user.
  • Unilaterally by the user after the expiration period has passed.

This script resembles certain two-factor wallets with a time-lock escape clause. The second clause ensures that users are able to move their funds if the auctioneer is unavailable.

Many interactions in CLM are based around accounts:

  • Fees paid to the auctioneer are deducted from your account
  • Fees gained by selling channels are credited to your account
  • Funds used to open channels to others are deducted from your account

As an account is just a UTXO, anytime a batch is cleared in the auction (market made, channels bought+sold), your account is spent, and re-created in the same transaction.

Creating An Account

Creating an account has two parameters: the size of the account, and the expiry of an account. As you'll see below, both values can be adjusted at any time.

We can create an account using pool, like so:

🏔 pool accounts new --amt=50000000 --expiry_height=1773394
{
        "trader_key": "0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732",
        "outpoint": "c6f62c80095c98a57f2eef485a7ff06611f97dc856754cad330f4eeb538ff514:0",
        "value": 50000000,
        "expiration_height": 1773394,
        "state": "PENDING_OPEN",
        "close_txid": "0000000000000000000000000000000000000000000000000000000000000000"
}

It's also possible to specify a relative account expiry based on the current best block with the --expiry_blocks argument. As an example, if I wanted my account to expiry in 2 weeks, I would pass: --expiry_blocks=2016.

Here I created an account with 0.5 BTC, that'll expire at height 1773394. The response shows that it's now pending open (unconfirmed), my trader_key (used to sign orders), and the outpoint of my new account.

Once at least 3 blocks have passed (in the alpha), the account will be confirmed and ready for use:

🏔 pool accounts list
{
        "accounts": [
                {
                        "trader_key": "0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732",
                        "outpoint": "c6f62c80095c98a57f2eef485a7ff06611f97dc856754cad330f4eeb538ff514:0",
                        "value": 50000000,
                        "expiration_height": 1773394,
                        "state": "OPEN",
                        "close_txid": "0000000000000000000000000000000000000000000000000000000000000000"
                }
        ]
}

Depositing To An Account

We can add more funds to an account using the pool accounts deposit command. Under the hood, we can actually batch other transactions with account modifications (make other payments, etc), but for now we expose only the basic functionality over the CLI.

NOTE: You should never send coins directly to your account output as it won't be picked up by the auctioneer.

Let's say I want to deposit an extra 1 million satoshis into my account, I can do so with the following command:

🏔 pool accounts deposit --trader_key=0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732 --amt=1000000 --sat_per_vbyte=5
{
        "account": {
                "trader_key": "0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732",
                "outpoint": "fef5cc4936290c6d57cda83bc3e90e75270296da8f34951cd562ac4cd37d4eef:0",
                "value": 50001714,
                "expiration_height": 1773394,
                "state": "PENDING_UPDATE",
                "close_txid": "0000000000000000000000000000000000000000000000000000000000000000"
        },
        "deposit_txid": "fef5cc4936290c6d57cda83bc3e90e75270296da8f34951cd562ac4cd37d4eef"
}

I specify my trader_key explicitly, as it's possible for poold to manage multiple accounts. The response shows my modified account, alongside with the txid that'll be used to service the deposit. Once this transaction has confirmed, I'll be able to use my account again.

Note that these funds came from the backing lnd node that poold is connected to. At a future time we also plan to support a traditional deposit address as well.

Withdrawing From An Account

Incrementally withdrawing from an account is also supported. The command is similar to the deposit command. If I wanted to extract that 1 million from that account (let's say it's my profit for the past week) and send elsewhere, I can do so with the following command:

🏔 pool accounts withdraw --trader_key=0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732 --amt=1000000 --sat_per_vbyte=5 --addr=tb1qe3ueyx8jhlj4h0s6mgywmtl8vlwxqkgkgp3m3s
{
        "account": {
                "trader_key": "0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732",
                "outpoint": "31664dcf5dd4e89a398e816afa55a36f7518560de08b3167d75bbc6804674cd1:1",
                "value": 49000801,
                "expiration_height": 1773394,
                "state": "PENDING_UPDATE",
                "close_txid": "0000000000000000000000000000000000000000000000000000000000000000"
        },
        "withdraw_txid": "31664dcf5dd4e89a398e816afa55a36f7518560de08b3167d75bbc6804674cd1"
}

Closing An Account

Finally, if you wish to send all your funds elsewhere, it's possible to close your account out before the main expiration period. We can close out the account we created above with the following command:

🏔 pool accounts close --trader_key=0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732

Orders

Now that we have our account set up and funded, it's time to trade some channels!

There're two types of orders in the current version of Pool: asks, and bids. You submit an ask when you have some coins that you want to lease out as inbound liquidity for a maximum period of time (expressed in blocks), at a fixed rate compounded per block. You submit a bid when you need to acquire inbound liquidity (ability to receive), for a minimum amount of time (again expressed in blocks), paying out a fixed rate that compounds per-block.

In the alpha version of Lightning Pool, a single lump sum premium is paid after order execution. In future versions, we plan on introducing "coupon channels" which allow for streaming interest to be paid out.

One important aspect of the market is that rather than buy/sell satoshis, we use units. A unit is simply 100,000 satoshis and represents the smallest channel that can be bought or sold on the network.

With that said, let's place some orders to try to earn some yield from this 0.5 BTC that's been burning a hole in our SD card for the past year. We'll place a single order for 10 million satoshis, wanting to receive 0.3% (30 bps) over a 3000 block period (a bit under 3 weeks):

🏔 pool orders submit ask 10000000 0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732 --interest_rate_percent=0.3 --max_duration_blocks=3000
-- Order Details --
Ask Amount: 0.1 BTC
Ask Duration: 3000
Total Premium (yield from taker): 0.0003 BTC
Rate Fixed: 1000
Rate Per Block: 0.000001000 (0.0001000%)
Execution Fee:  0.00010001 BTC
Max batch fee rate: 25000 sat/kw
Max chain fee: 0.016325 BTC
Confirm order (yes/no): yes
{
        "accepted_order_nonce": "f1bebca6047dee6657f82377ebac94d1dc6667097f2a4d463deb63eff6f0dbcf"
}

By leaving off the --force flag, we request the final break down to confirm the details of our order before we put it through.

In this case, if this order is executed, then I'll gain 30k satoshis:

premium = (rate_fixed / billion) * amount * blocks
30,000 = (1,000/1,000,000,000)*1,000,000*3,000

It's important to note that although internally we use a fixed rate per block to compute the final premium, on the command line, we accept the final acceptable premium as a percentage. Therefore, when submitting orders, one should place the value that they wish to receive or accept at the end of the lease period. Internally, we'll then compute the per block lease rate and submit the order using that.

The duration and fixed rate (the percentage) are two important values to pay attention to when placing orders. Given the same amount, and fixed rate, you earn more by leasing out the funds for a longer period of time. Conversely, a taker will pay more if they need the funds for a longer period of time.

Also notice the +max batch fee* break down, that regulates the highest chain fee you're willing to pay to get into a batch. When traders are included in a batch, they split the channel open fee with the party they're matched with, then pay for their account to be spent and re-created. The auctioneer then uses this value during match making to ensure that traders don't pay more chain fees than they intend to. If your desired chain fee is below the current proposed batch chain fee, then your order won't be eligible for execution until chain fees come down somewhat.

Users can use the --max_batch_fee_rate value to regulate chain fees. Note that the values is expressed in sat/kw on the command line. To convert from sat/vbyte to sat/kw, simply divide by 250.

Take note of the order_nonce, it's used through the auction to identify orders, and also for authentication purposes.

We can then check out the order we just placed with the following command:

🏔 pool orders list
{
    "asks": [
    {
        "details": {
                "trader_key": "0288096be9917f8ebdfc6eb2701635fe658f4eae1e0274dcce41418b3fb5145732",
                "rate_fixed": 1,
                "amt": "10000000",
                "funding_fee_rate": "253",
                "order_nonce": "f1bebca6047dee6657f82377ebac94d1dc6667097f2a4d463deb63eff6f0dbcf",
                "state": "ORDER_SUBMITTED",
                "units": 100,
                "units_unfulfilled": 100
        },
        "max_duration_blocks": 3000,
        "version": 0,
    },
}

The order hasn't been cleared yet (state ORDER_SUBMITTED), and it shows up as 100 units, or 10 million satoshis.

If we instead wanted to buy inbound bandwidth, we could submit a bid instead. A trader can have multiple unfilled bids and asks. Partial matching is possible as well, so someone could only purchase 10 of the 100 units we have for sale. Over time the orders will gain additional constraints such as fill-or-kill, or min partial match size.

Channel Leases

Once an order has been matched in an auction, the pool auction leases command can be used to examine your current set of purchased/sold channel leases. An example output looks something like the following:

🏔 pool auction leases
{
        "leases": [
                {
                        "channel_point": "78cc6879c1dc1c00f22b29a06458f1335ed0fdb7d05c01b9077e3155e697bbb9:2",
                        "channel_amt_sat": 5000000,
                        "channel_duration_blocks": 144,
                        "premium_sat": 40000,
                        "execution_fee_sat": 5001,
                        "chain_fee_sat": 165,
                        "order_nonce": "eb972cd21cf1651e251c8b07d69e89c47294bae104fbdc9da26edf9aee335c9a",
                        "purchased": false
                }
        ],
        "total_amt_earned_sat": 40000,
        "total_amt_paid_sat": 5166
}

Here we can see I sold a channel for 40k satoshis, and ended up paying 5k satoshis in chain and execution fees, netting a cool 35k satoshi yield. Within the actual auction, these numbers will vary based on the chain fee rate, the market prices, and also the execution fees. Users can constraint how much chain fees they'll pay by setting the --max_batch_fee_rate argument when submitting orders.

Batched Uniform-Price Clearing

Now that we have orders submitted, how does the rest of the auction actually work? As mentioned above, Pool conducts a discrete batch auction every 10 minutes. This is distinct from regular continuous exchanges in that orders are only cleared every 10 minutes. Orders are also sealed-bid, meaning that other traders in the venue are unable to see what others have bid. On top of this, we utilize a uniform-clearing price algorithm to give all traders in the batch the same interest rate. This is the same mechanism used by the U.S Treasury for its bonds, and is intended to promote fairness as your order will only be matched with a price better than your initial ask/bid.

Note that it's possible that after the 10 minutes interval has passed a market can't be made (supply and demand didn't cross). In this case, nothing happens, and we just wait for the next batch to come across.

To illustrate how the uniform price clearing works consider the following example. Let's say I want to buy 100 million satoshis (1 BTC, 1000 units), for at least 10 days (1440 blocks) at a price of 5% (using high numbers to make it easy to follow). However, the market clearing price (where the supply+demand curves cross) is actually 1%. In this case I bid more than the market clearing price, but end up paying that price, as it's the best price that was possible in that market.

A simple rule of thumb for bids and asks is as follows:

  • When I submit a bid, I'll either pay that amount or less.
  • When I submit an ask, I'll either receive that amount or more.

All orders in a batch are executed in a single on-chain transaction. This allows for thousands of channels to be bought/sold atomically in a single block. We call the transaction that executes the orders the Batch Execution Transaction.

The pool auction sub-command houses a number of useful commands to explore the past batches, and examine the current auction parameters.

One can browse the latest cleared batch using the pool auction snapshot command:

🏔 pool auction snapshot 
{
        "version": 0,
        "batch_id": "02824d0cbac65e01712124c50ff2cc74ce22851d7b444c1bf2ae66afefb8eaf27f",
        "prev_batch_id": "03687baa3c7414e800ddba37edacb3281999739303b7290a69bd457f428ecd9b2c",
        "clearing_price_rate": 976,
        "matched_orders": [
                {
                        "ask": {
                                "version": 0,
                                "max_duration_blocks": 4032,
                                "rate_fixed": 744,
                                "chan_type": 0
                        },
                        "bid": {
                                "version": 0,
                                "min_duration_blocks": 1024,
                                "rate_fixed": 976,
                                "chan_type": 0
                        },
                        "matching_rate": 976,
                        "total_sats_cleared": "1000000",
                        "units_matched": 10
                }
        ],
        "batch_tx_id": "4508169e371096ad85e57f251e7b0034910a5e4799f3e9714d7df98f85fd8b93",
        "batch_tx": "0200000000010307368f8721608c58743c452562b4fb300f3a983e0ce32e16975236493de64b4a01000000000000000019947c40c19c14f3e0ba9795c80e878e5ac9d19513f95fb63204590603c78e7a000000000000000000c6bbf036ec29cc79efa15e4b779ae33286ddf3d0ff06fba720b58e9652f030bb010000000000
0000000440420f0000000000220020169c54346374ed74d0654d4fc6fa493c637cdd8ce7c76ad24a476e7d370b926697490f0000000000220020c13828d72d6a3fd12e939d46153ebcb2cc1c7bbb0958d53d92701ba1ba5930eb0bbbc901000000002200201ec50230e41f0f0978e1b0c475bfe8af1e032135b65233a86affd9d56b320f6e99adeb020
000000022002026e0d02777ed45059d70233dfdec0aa30abe40fdc26fad5aa780f9448a399118024730440220641ad6ef4d754ad7e6164c9743b549d194db1b0a1d4fd1c4c8f47b6e044203e402206db731b2b0eebd9244f27118aaeb85bd7679769eac57cba395837a3c8b4ff24101232103ba06cff976b410f9381f297d9693544a19c504527f5a4c
c0eb2966b3900343b6ac0347304402207b0344aa98878e5aa40dc0fb712beff9b11d7fba3671f847996d83b4f6a643f90220720aa47f0ac229e4d14eebe38cffbdb2a344241e56151a0ce057f9c4cc001a1201483045022100be8808e71b6867521ed16c7749d78fb809ea0fc72f33d3b2b752cb4a13bc4ad802202335fda42f030a058003437dc7e05
39a6d36f3ce94045e20f3e176f165bc5ef0014e2103d9ebf3cea856f88ee98801621b7ea837951c530f69bc26da94d58f13417a4993ad2103a6051079a5910dd7c8d055b6713bdc0370e4983ee048a7ae26d9c52f7321949fac7364038b341bb1680347304402202ed63c0225afc718169c081b33e1bb2049cee8126539275ad62afcdf17adf74a0220
04d6c18d9a98e60642e4665428cccd71ebbc2e30fe81aee5b2bf10d682875dc901483045022100ba598f8480ed6dcdbdda1e30166b43a86f72bbc23d20bfe8751553bc8ecc6a3f02203412399095fd1429b924bfe224b64f0840172686a8af9dd3b18dc4ed40de1e23014e21038be01624676bf63a9d7d829175a70193a7e8680452b9b192ec6cf6654
a7e3be1ad2103bc6202b694e62a4d890cbb83f3a4dddb964fc500b25f55a38501642a770e3f37ac7364038a341bb16800000000"

Here we see a batch where a single order was matched, at a clearing rate of 976, with a single channel being purchased with a lifetime of 1024 blocks, or roughly one week.

Note that the pool auction snapshot command can be used to determine the past marker clearing price, which can be useful when deciding what your bid/ask should be. There's no explicit "market buy" function, but submitting a bid/ask at a similar clearing_price_rate is equivalent.

The command also accept a target batch_id as well. Here we can use the prev_batch_id to examine the prior batch, similar to traversing a link-listed/blockchain:

🏔 pool auction snapshot --batch_id=03687baa3c7414e800ddba37edacb3281999739303b7290a69bd457f428ecd9b2c

Service Level Lifetime Enforcement

In the alpha version of Pool, script level enforcement isn't yet implemented. Script level enforcement would lock the maker's funds in the channel for the lease period. This ensures that they can't just collect the premium (before coupon channels) and close out the channel instantly. With script enforcement, they would be able to close the channel (force close it), but their funds would be unavailable until the maturity period has passed.

Instead, we've implemented a feature in lnd to prevent channels from being cooperatively closed by the maker until the expiry height (what we call the thaw_height). Additionally, if we detect a force close by the maker of that channel, then we'll ban them from the market for a set period of time.

Prerequisites

To build and run LLM, the following tools/binaries must be installed on your system:

  • Go 1.13 or later
  • make
  • lnd version v0.11.1-beta or later with the build tags signrpc, walletrpc, chainrpc and invoicesrpc enabled (or the current master branch built with make install tags="signrpc walletrpc chainrpc invoicesrpc" if versionv0.11.1-beta is not yet released).

Installation

The following section assumes at least Go 1.13 is installed.

To install the primary daemon (poold) and the CLI used to control the daemon, the following command should be run:

🏔  make install

Assuming poold is now in your $PATH, you can start the daemon with the following command (assuming you have a local testnet lnd running):

🏔 poold --network=testnet --debuglevel=trace

The current server is reachable at clm.testnet.lightningcluster.com:12010, this may change as the alpha version progresses.

Authentication and transport security

The gRPC and REST connections of poold are encrypted with TLS and secured with macaroon authentication the same way lnd is.

If no custom base directory is set then the TLS certificate is stored in ~/.pool/<network>/tls.cert and the base macaroon in ~/.pool/<network>/pool.macaroon.

The pool command will pick up these file automatically on mainnet if no custom base directory is used. For other networks it should be sufficient to add the --network flag to tell the CLI in what sub directory to look for the files.

For more information on macaroons, see the macaroon documentation of lnd.

NOTE: pool's macaroons are independent from lnd's. The same macaroon cannot be used for both poold and lnd.