Haskoin Store

A Bitcoin (BTC) and Bitcoin Cash (BCH) block chain indexer with a REST API, written in Haskell. It connects to the peer-to-peer network directly, so it does not need a full node next to it, and keeps its indices in RocksDB.

• API reference
• Library documentation and client documentation
• Changelog
• Report a bug

Contents

• Why Haskoin Store
• Design principles
• Features
• Quick start
• Building from source
• Running
• Tutorial: following a wallet
• Configuration
• API
• How it works
• Monitoring
• Using it from Haskell
• Packages
• Development
• Related projects
• License

Why Haskoin Store

A Bitcoin node can tell you what is in block 800000, but not what the balance of an address is, which transactions belong to a wallet, or which outputs it can spend. Wallets, block explorers, payment processors and accounting tools all need exactly those answers, for many users at once, in milliseconds.

The usual way to get them is to run a full node and an indexer next to it, and operate both. Haskoin Store is one program instead. It connects to the peer-to-peer network itself, downloads blocks and unconfirmed transactions, keeps indices by address, transaction and extended public key in RocksDB, and serves them over a REST API. It has been used to serve public Bitcoin and Bitcoin Cash APIs such as api.haskoin.com.

Design principles

• Index for the questions people ask. Every index is laid out so that an API query is one seek and a short scan in RocksDB, newest first, which keeps response times flat as the chain grows.
• Never half-write. Each block, and each batch of mempool transactions, is one atomic database write. A crash leaves the store at the previous block, with no repair step on restart.
• Unconfirmed transactions matter. The mempool is indexed and persisted like the chain, ordered before confirmed history, and handles replace-by-fee on Bitcoin. A wallet sees a payment the moment the network does.
• Extended public keys are first class. Wallets are mostly xpubs, so the store derives their addresses up to an exact gap, remembers the derivations, and can keep large wallets ready in a Redis shared by several stores.
• Bounded resources. API limits stop one request from monopolising the server, slow event clients are dropped, and RocksDB memory has a budget.
• Operable. A health endpoint that tells a load balancer when an instance is behind, StatsD metrics for everything, configuration through flags or environment variables, and a Docker image.
• Follow the work, not a node. The store trusts the chain with the most proof of work, as reported by haskoin-node, and checks what it needs to keep its indices consistent. It does not re-validate scripts; run it against the network you trust, or next to your own node with --peer.

Features

Area	What you get
Networks	Bitcoin and Bitcoin Cash: mainnet, testnets and regtest
Addresses	Balances, transactions and unspent outputs (UTXO) of any address, alone or many at once
Xpubs	Balances, transactions and UTXO of extended public keys, for legacy, nested SegWit and native SegWit addresses, with an optional Redis cache for large wallets
Blocks and txs	Lookup by hash, height or time; raw serialized forms; transactions of a block
Mempool	Persistent, indexed by arrival time, with replace-by-fee on BTC and pruning of old entries
Formats	JSON, pretty JSON, or a compact binary encoding
Live events	New blocks and transactions as a JSON stream or over a WebSocket
Compatibility	Endpoints in the style of Blockchain.info under /blockchain/
Operations	Health check, StatsD metrics, RocksDB memory limits, Docker image

What is not included

• No wallet: it never holds keys or signs anything.
• No full validation of scripts and consensus rules. It follows the chain with the most work and indexes what peers send.
• No address or transaction search by anything but its hash; no indices on scripts without an address.

Quick start

With Docker:

git clone https://github.com/jprupp/haskoin-store.git
cd haskoin-store
docker build -t haskoin-store .
docker run -d --name haskoin-store -p 3000:3000 \
  -e NET=btc -e DISCOVER=true \
  -v haskoin-store-data:/home/haskoin/.haskoin-store \
  haskoin-store
curl 'http://localhost:3000/block/best?pretty=true'

The first sync downloads and indexes the whole chain, which takes a long time and a lot of disk. The API answers while it syncs; /health reports whether the node is in sync.

Building from source

You need Stack and three C libraries: zlib, libsecp256k1 and RocksDB. Stack installs the right GHC by itself.

Debian or Ubuntu

sudo apt install git pkg-config zlib1g-dev libsecp256k1-dev librocksdb-dev haskell-stack

Fedora

sudo dnf install git pkgconf zlib-devel libsecp256k1-devel rocksdb-devel stack

Then build and install the haskoin-store binary to ~/.local/bin:

git clone https://github.com/jprupp/haskoin-store.git
cd haskoin-store
stack build --copy-bins
~/.local/bin/haskoin-store --help

Any distribution, with Nix: Stack fetches the C libraries listed in stack.yaml from Nix instead of the system.

stack --nix build --copy-bins

Running

haskoin-store --net btc --discover

Give it either --discover or at least one --peer; with neither it turns discovery on by itself. Data goes in a directory named after the network inside --dir, so one data directory can hold several networks.

For a big public deployment, also set MALLOC_ARENA_MAX=2 (the Docker image does) to keep glibc from fragmenting RocksDB's memory.

Tutorial: following a wallet

This walks through what a wallet back end typically does with the API, on a store that is already running on localhost:3000.

1. Wait for the store to be ready

curl -s -o /dev/null -w '%{http_code}\n' localhost:3000/health

200 means the index is caught up with the network. While it syncs you get 503, and the body says which check fails.

2. Get the balance of a wallet

Most wallets are an extended public key. Ask for its summary, and say which kind of addresses it derives (legacy, the default, compat for nested SegWit, segwit for native SegWit):

curl 'localhost:3000/xpub/xpub6C...?derive=segwit&pretty=true'

The answer has confirmed and unconfirmed balances, and the index of the last used receiving and change address, so the wallet knows where to derive the next address.

If you get derive wrong, the store says so: a wallet with no transactions of the kind you asked for, but some of another kind, gets a 400 whose message names the right one, instead of an empty wallet that looks like a zero balance.

3. List its transactions, page by page

curl 'localhost:3000/xpub/xpub6C.../transactions?derive=segwit&limit=50'

Results are newest first, unconfirmed ones on top. To get the next page, start from the last transaction you received instead of using an offset:

curl 'localhost:3000/xpub/xpub6C.../transactions?derive=segwit&limit=50&offset=1&height=<last txid>'

offset=1 skips the transaction you already have. Paging by starting point stays fast however deep you go, and does not shift when new transactions arrive.

4. Find spendable outputs

curl 'localhost:3000/xpub/xpub6C.../unspent?derive=segwit'

Each output comes with its derivation path, so the wallet knows which key signs it.

5. Broadcast a payment

curl -X POST --data-binary @tx.hex localhost:3000/transactions

The body is the signed transaction in hex. The store sends it to its peers and returns the transaction id. It appears in the mempool, and in the wallet's history, as soon as a peer relays it back.

6. Stay up to date without polling

curl -N localhost:3000/events

Each line is a new block or a mempool change. A wallet back end keeps this open and refreshes the affected wallets when an event arrives, instead of asking every few seconds.

After a lost connection, come back with the hash of the last event you heard, and the stream starts with what came after it:

curl -N "localhost:3000/events?after=9a5c…"

The server keeps its last 10,000 events. It answers 404 when it no longer has the one you name: read your wallets again, and listen without after.

Over a WebSocket at the same path, a client can ask for only the transactions it cares about. It sends one or more of these, as text:

{"type":"watch","addresses":["bc1q…","1A1z…"]}
{"type":"watch","xpub":"xpub6C…","derive":"segwit"}
{"type":"filterload","filter":"b50f","functions":11,"tweak":0,"flags":0}

From then on it hears of a transaction only when it touches one of the addresses, an address of the key, or matches the Bloom filter (BIP 37, as SPV wallets use). Blocks are always sent. A key is followed as a wallet uses it: the gap of unused addresses moves on as payments come. For a key with several types of address, send one message for each type.

A client that connects and then sends its messages gets every transaction until the first message is read. To start with a choice, put it in the URL, of the WebSocket or of the HTTP stream:

curl -N "localhost:3000/events?quiet=true"                  # blocks only
curl -N "localhost:3000/events?addresses=bc1q…,1A1z…"
curl -N "localhost:3000/events?xpub=xpub6C…&derive=legacy,segwit"

quiet=true sends blocks alone until the client watches something; a WebSocket client with a long list connects with it and sends the list in messages. The API reference has every message and parameter, and the binary form of each message.

7. Make big wallets fast

Start the store with --redis and a Redis (or Valkey) server. Xpubs with at least --redis-min-addrs used addresses, or at least --redis-min-txs transactions in their addresses, are then cached after their first query and answered from Redis, transactions in full included. Several stores can share one Redis: they keep each other's cache up to date, and a store that has not imported a transaction yet still serves it from the cache. Add nocache=true to a query to read the database instead.

Servers behind a load balancer should also be started with --fleet (A fleet of servers): they then answer every request alike, and share a cache of xpubs that is built for that.

Configuration

Every option has an environment variable that sets its default, which is handy in containers. Booleans in variables accept true/false, yes/no, on/off and 1/0. haskoin-store --help shows the current values.

Network and peers

Option	Environment	Default	Meaning
-n, --net	NET	bch	Network: btc, bch, or a test network (see --help)
-d, --dir	DIR	~/.haskoin-store	Data directory
--discover	DISCOVER	off	Discover peers through DNS seeds and relayed addresses; on regtest it finds none, so name a --peer
-p, --peer	PEER	none	Connect to HOST[:PORT]; repeat for more, space-separated in PEER
--max-peers	MAX_PEERS	20	Most peers to connect to
--peer-timeout	PEER_TIMEOUT	120	Seconds before an unresponsive peer is dropped
--max-peer-life	MAX_PEER_LIFE	172800	Seconds after which a peer is dropped, so connections rotate
--block-window	BLOCK_WINDOW	64	Blocks to request from the syncing peer at a time

Mempool

Option	Environment	Default	Meaning
--no-mempool	NO_MEMPOOL	off	Do not index unconfirmed transactions
--sync-mempool	SYNC_MEMPOOL	off	Ask each peer for its whole mempool once in sync
--wipe-mempool	WIPE_MEMPOOL	off	Empty the mempool area of the database at start
--mempool-timeout	MEMPOOL_TIMEOUT	14	Days after which unconfirmed transactions are deleted (0 = never)

Web server and limits

Option	Environment	Default	Meaning
--host / --port	HOST / PORT	* / 3000	Address the API listens on
--noweb	NO_WEB	off	Index only, no API
--tls-cert / --tls-key	TLS_CERT / TLS_KEY	none	Serve HTTPS with this certificate chain and private key, both files in PEM
--no-xpub	NO_XPUB	off	Disable the xpub endpoints
--no-blockchain-info	NO_BLOCKCHAIN_INFO	off	Disable the /blockchain/ endpoints and price lookups
--max-item-count	MAX_ITEM_COUNT	200000	Most items in a list (0 = no limit)
--max-full-item-count	MAX_FULL_ITEM_COUNT	5000	Most full transactions in a list (0 = no limit)
--max-offset	MAX_OFFSET	50000	Largest offset (0 = no limit)
--def-item-count	DEF_ITEM_COUNT	100	Items in a list when no limit is given
--xpub-gap	XPUB_GAP	32	Unused addresses kept after the last used one of each chain of an xpub; payments further away are not seen
--xpub-gap-init	XPUB_GAP_INIT	20	Addresses looked at for an xpub with no transactions; an xpub with none used among them counts as unused
--ticker-url	TICKER_URL	Blockchain.info ticker	Source of current prices
--price-history-url	PRICE_HISTORY_URL	Blockchain.info price index	Source of historical prices
--ticker-refresh	TICKER_REFRESH	90000000	Microseconds between price updates

Redis cache

Option	Environment	Default	Meaning
--redis	REDIS	off	Cache large xpubs in Redis
--redis-url	REDIS_URL	local Redis	Redis URL, such as redis://host:6379
--redis-min-addrs	REDIS_MIN_ADDRS	100	Cache xpubs with at least this many used addresses
--redis-min-txs	REDIS_MIN_TXS	1000	Or with at least this many transactions in their addresses
--redis-max-xpubs	REDIS_MAX_XPUBS	10000	Evict the least recently used xpubs past this many

Every key of the cache starts with hs2:<network>:, so a Redis can be shared by stores of several networks, and by stores of older versions, whose cache uses other keys. --redis-max-keys and --redis-sync-interval of older versions are gone.

A fleet of servers

Option	Environment	Default	Meaning
--fleet	FLEET	off	Answer as one of the servers that share the Redis of --redis

Servers behind a load balancer import the same blocks and transactions at slightly different times, so the same request can get different answers depending on which of them it reaches. Start every one of them with --redis, the same --redis-url and --fleet, and they all answer from the same block and the same mempool:

• The block is the newest that every live server has. Each server imports blocks when it gets them and keeps its database as it was at each of its last six, and a request is answered from the block that the fleet agreed on. The fleet moves to a new block in one step, when the last server has it.
• The mempool is one pool in Redis, which every server offers its unconfirmed transactions to and all answer from. A transaction has one time of arrival there, so lists come in the same order everywhere.
• A server that falls behind answers 503 with behind-fleet rather than something that the others would not say, and stops counting after ten seconds without a report, so that it holds nobody up. A server whose health check fails stops reporting.
• /events is the same stream on every server. What changes the shared mempool or moves the fleet to a block writes an event to Redis in the same step, and every server sends those events, in that order, instead of what its own node saw. A block is told when the whole fleet can answer about it. Each server keeps the same last events, so a client that reconnects with after= may land on any of them.
• Without Redis each server answers from its own database, and /health says 203, as it does for the cache.
• Large xpubs are cached for the whole fleet, under the same --redis-min-addrs, --redis-min-txs and --redis-max-xpubs as the cache of a server alone, which should be the same on every server. An entry holds the chain's part of a wallet at the block that the fleet answers from, and the pool's part is kept by the pool itself, so a cached answer is the one that the database gives, on every server. The server that moves the fleet to a new block brings the cache there; in the moment before it has, and for a wallet that just gained or lost an address, requests are answered from the database. A reorganisation empties the cache. With it a request about a wallet of 464 addresses and 19,499 transactions takes 1.3 to 2.2 milliseconds, where it took 5 to 38.

The rules, and the tests that hold Redis to them, are described in "A fleet that answers as one" in docs/layered-database.md.

Health check

Option	Environment	Default	Meaning
--max-lagging-blocks	MAX_LAGGING_BLOCKS	3	Most blocks the index may lag behind the headers
--max-pending-txs	MAX_PENDING_TXS	10000	Most transactions waiting to be imported
--min-peers	MIN_PEERS	1	Fewest connected peers
--block-timeout	BLOCK_TIMEOUT	14400	Most seconds since the last block
--tx-timeout	TX_TIMEOUT	1800	Most seconds since the last transaction
--health-check-interval	HEALTH_CHECK_INTERVAL	30	Seconds between checks

Metrics and logging

Option	Environment	Default	Meaning
--statsd	STATSD	off	Send metrics to StatsD
--statsd-host / --statsd-port	STATSD_HOST / STATSD_PORT	localhost / 8125	StatsD server
--statsd-prefix	STATSD_PREFIX	haskoin_store	Prefix of every metric
--statsd-no-events	STATSD_NO_EVENTS	off	Do not send individual samples
--statsd-aggregates	STATSD_AGGREGATES	off	Also send aggregated statistics every second
--debug	DEBUG	off	Verbose logging

RocksDB

Option	Environment	Default	Meaning
--no-bloom	BLOOM	on	No Bloom filters on what is read by key; BLOOM=false is the same
--max-open-files	MAX_OPEN_FILES	2000	Open files; bounds index and filter memory (0 = unlimited)
--write-buffer-mb	WRITE_BUFFER_MB	256	Memtables across all column families (0 = unlimited)
--max-wal-mb	MAX_WAL_MB	1024	Write-ahead log size that forces flushes (0 = RocksDB default)
--block-cache-mb	BLOCK_CACHE_MB	1024	Shared block cache, holding index and filter blocks too (0 = RocksDB default)

Memory used by RocksDB is roughly the block cache plus the memtables, on top of the Haskell heap. /dbstats shows where it goes.

Bloom filters are on by default. They let RocksDB skip the table files that cannot have a key, which is most of what a lookup of a transaction, an output or a balance would otherwise read.

• What has them: the data that is read by key: transactions, spenders, unspent outputs, blocks, balances and the mempool. The two indices by address, which are scanned, do not.
• What they cost: about 1.25 bytes for each key, held in the block cache. On a chain with billions of keys that is gigabytes: size --block-cache-mb for it, or turn them off with --no-bloom.
• An existing database gets filters for the table files that are written from now on, so their memory comes gradually, as RocksDB compacts. The migration to database version 22 rewrites nearly every transaction, and with it most of the filters arrive at once.
• --bloom was the way to turn them on, and is still accepted. Up to version 1.8.1 it built no filters at all on RocksDB 6.24 or later, because of a wrong type in the binding, fixed in rocksdb-haskell-jprupp 2.6.3.

API

Full reference: swagger.yaml, also served at api.haskoin.com. It explains the conventions of the API (amounts, ordering, paging, formats, errors, unconfirmed transactions and reorganisations) and the reasons for them, as well as every endpoint.

curl localhost:3000/address/bc1q…/balance
curl localhost:3000/xpub/xpub6C…/balances
curl localhost:3000/transaction/<txid>
curl -N localhost:3000/events              # stream of new blocks and txs

• The root of the server, /, is the reference of its API as a page that can try requests, and /swagger.yaml the file behind it.
• Add ?pretty=true for indented JSON.
• Send Accept: application/octet-stream for the binary format: see Binary format below.
• Lists are paged with limit and offset, newest first. height begins a page at a transaction hash, a block hash, a block height or a Unix time instead.
• Parameters are checked: a parameter that an endpoint does not accept, or a value it cannot read, gives 400 with a message that names it, instead of being ignored. The Blockchain.info endpoints accept format=json, which clients written for them often send.
• Errors come back with an HTTP status, in the format that was asked for: 400 for bad input, 404 for unknown things, 409 when a short transaction number is ambiguous, 413 for requests that ask for too much.

HTTP and HTTPS

• HEAD and OPTIONS are answered, the second with what a browser may send from another site.
• Answers are compressed for a client that sends Accept-Encoding: gzip, but for the stream of events.
• Long lists go in a body. The endpoints that take a list of addresses, blocks, heights or transactions also take it under POST, as a form (application/x-www-form-urlencoded), so that it does not have to fit in a URL. /transactions does not: POST there publishes a transaction.
• A 503 says when to try again, in Retry-After.
• HTTPS is optional and off by default. With --tls-cert and --tls-key the server speaks HTTPS on its port, and nothing else. A public service usually has a proxy in front that holds the certificate; this is for a server by itself. The files are read at start, so a renewed certificate takes a restart.

Binary format

Every endpoint but those under /blockchain/ and /dbstats also answers in a compact binary format, which is smaller and faster to decode than JSON. The Haskell client in haskoin-store-data speaks it, and swagger.yaml has a section, "Binary format", that is enough to write a client in another language: the primitives, the layout of every type, and what each endpoint returns.

• Ask for it with Accept: application/octet-stream. In a list of types it wins when its q weight is above 0 and no less than that of application/json; a wildcard or no header gives JSON. The answer has Content-Type: application/octet-stream, and every answer has Vary: Accept.
• Errors come in binary too: one byte that says which error, and a message or a list of transactions for those that have one. Up to version 1.8.1 they were JSON whatever was asked, so a client that also talks to older servers reads the Content-Type of an error.
• Lists start with their length, the raw lists of blocks and transactions included, which up to version 1.8.1 had none.
• /events is the exception: a stream of events of 33 bytes each, with no count, for as long as the connection is open.
• Over the WebSocket binary is binary frames, for the events and for the commands that choose transactions. The server answers in the kind of frame that it was last sent.
• Everything is big-endian, except what is copied from the peer-to-peer protocol, such as a whole transaction or block.

Endpoints at a glance

Area	Endpoints
Blocks	/block/best, /block/latest, /block/:hash, /blocks, /block/height/:h, /block/heights, /block/time/:t, /block/mtp/:t, each with a /raw variant where it makes sense
Transactions	/transaction/:txid, /transactions, /transactions/block/:hash, /transaction/:txid/after/:height, POST /transactions, /raw variants
Mempool	/mempool
Addresses	/address/:addr/{transactions,transactions/full,balance,unspent} and the multi-address forms /address/{transactions,transactions/full,balances,unspent}?addresses=…
Xpubs	/xpub/:xpub, /xpub/:xpub/{transactions,transactions/full,balances,unspent}, DELETE /xpub/:xpub (drop from cache); derive=legacy|compat|segwit picks the address type (normal, the old name of legacy, still works)
Network	/peers, /health (503 when unhealthy, 203 without its Redis), /dbstats
Events	/events, as JSON lines over HTTP or as a WebSocket; after= resumes, and a WebSocket client can filter by address, xpub or Bloom filter
Blockchain.info	/blockchain/multiaddr, /blockchain/balance, /blockchain/rawaddr/:addr, /blockchain/unspent, /blockchain/rawtx/:txid, /blockchain/rawblock/:hash, /blockchain/latestblock, /blockchain/unconfirmed-transactions, /blockchain/export-history, /blockchain/q/…

Turn off the xpub endpoints with --no-xpub and the Blockchain.info ones with --no-blockchain-info.

How it works

peers ──► haskoin-node ──► block store ──► RocksDB ──► web server ──► clients
                │               │             ▲
                └─ headers      └─ events ──► cache ◄──► Redis ◄──► other stores

• haskoin-node keeps a pool of peers and syncs block headers.
• The block store (Haskoin.Store.BlockStore) downloads blocks from one peer at a time and mempool transactions from all of them, and imports them with the rules in Haskoin.Store.Logic.Layered. Each import is one atomic RocksDB batch, so a crash never leaves a half-written block.
• Reorganisations are handled by reverting blocks until the best block is back on the main chain of headers.
• The cache (Haskoin.Store.Cache) keeps xpubs with many used addresses or many transactions in Redis: their balances, transactions, unspent outputs and the bodies of their transactions. It never computes an entry from what it guesses changed: when a block, a mempool transaction or a reorganisation touches a cached xpub, the store reads from RocksDB, with the same queries that answer uncached requests, the transactions that changed and the outputs they create and spend, and writes the differences in one Lua script, together with every transaction body the entry needs that Redis does not have yet (bodies it has, such as those of the transactions a block confirms, are rewritten just before, in small scripts, so that a large block does not hold Redis for long). The time of a reload no longer grows with the size of the xpub. The store reads the whole xpub instead when it first caches it, and whenever it cannot be sure the entry has every earlier change: when the entry is more than a block behind, when an address with transactions enters or leaves the gap, when the balance of an address that no change touched differs from Redis's and its newest transactions do not explain why, and on a schedule, every five to ten minutes. The interval depends on the xpub, so that xpubs cached together are not all read whole at the same moment, which would hold them all in memory at once. Under load the database is often a little ahead of the events being applied, so a reload first adds the transactions that the database has for such an address since the block the entry was written at, and the mempool transactions of the entry that have since been confirmed or dropped. The script writes nothing if the entry changed since it was read, so the entry is always what the database says; and Redis, which answers one client at a time, is held for the size of the change, not of the xpub. A page of a cached xpub is one round trip to Redis. The Lua scripts are in src/Haskoin/Store/Cache/lua/, one file each, embedded in the library when it is compiled. Stores sharing a Redis claim each change, so it is applied once, by the first store to see it; a store never replaces an entry that knows more than its database (a higher block, or a transaction it has not seen), and a store that is catching up does no cache work. Without Redis, xpub queries read RocksDB directly. The ways an entry can still come to differ from the database unnoticed, such as a missed spend of an output of a cached transaction, and what would rule them out, are in docs/cache-consistency.md.
• A fleet (Haskoin.Store.Fleet.Member, with --fleet) has a cache of xpubs of another kind (Haskoin.Store.Fleet.XPub), because its servers do not answer from their own databases but from one block and one shared mempool. An entry has two layers: what the chain has for the addresses of the xpub at the block that the fleet answers from, and what the shared pool has for them, which the script of the pool writes as it changes the pool. The first follows from the chain, so any server writes the same, and nothing has to be claimed or repaired: the server that moves the fleet to a new block applies what the block did, and a cache that was not brought along is emptied. docs/layered-database.md has the design, the tests and the measurements.
• The web server (Haskoin.Store.Web) answers from RocksDB, and from the cache for cached xpubs and for transaction bodies that Redis has.

Xpub addresses are derived up to an exact gap: every address up to the last used one of each chain, then exactly --xpub-gap unused ones, and always the first --xpub-gap-init addresses. A payment further away is not seen, as a wallet following BIP 44 with the same gap would not see it; raise the gap to see it.

An xpub does not record which kind of address its wallet derives, so a request names it with derive. When an xpub has no transactions with the kind asked for, the store reads the first --xpub-gap-init addresses of the other kinds, and if one of them is used it refuses the request with 400 and names that kind: an empty answer would look like a wallet with no money, a mistake that is easy to miss. The extra reads happen only for wallets that look empty, and not at all on networks without SegWit, which have one kind of address.

The database layout is described in Haskoin.Store.Database.Types.

Unconfirmed transactions are kept in an area of their own, and applied on top of the chain when a query is answered, so that nothing the mempool does changes the chain's data. This is version 21 of the database, the layered layout of docs/layered-database.md, which has the reasons, the measurements and the evidence. Up to version 20 they were written into the same indices as the block chain. A database of version 20 is migrated the first time this version of the store opens it: its mempool is moved to the new area, in batches that can be interrupted, and the log says how many transactions were moved. There is no way back: an older store refuses the database afterwards, so take a copy of a production database first.

Version 22 makes the record of a confirmed transaction something that is written once, by the block that confirms it, and the same on every server that has that block.

• Its time is the time of its block. Until then a server that had seen a transaction in its mempool kept the time of that when a block confirmed it, so two servers gave two times for one confirmed transaction, and a server that synced later gave a third.
• The spenders of its outputs are under keys of their own, sixteen outputs to a key. Until then they were in the record, which was read and written whole again each time a block spent one of its outputs. Reading a confirmed transaction in full now takes a second key: 18 microseconds where it took 11.5, measured, and 0.8 milliseconds for a hundred where it took 0.4. Lists by address and by xpub do not read records and cost the same.

A transaction that leaves the mempool unconfirmed, because it was replaced, double-spent by a block or too old, is no longer kept: the store forgets it, and the API answers 404 for it, as it already did. Until version 22 its body stayed in the database, marked as deleted, for ever. The bodies that are there in a migrated database are left where they are and ignored, since finding them would mean reading every transaction.

A database of version 21, or of version 20 once its mempool is moved, is brought to this the first time this version of the store opens it. The store reads every transaction that it has, and writes nearly every one again, before it starts: nothing lists the records that need it, and most have a spender in them. The log says how far it is every million transactions. A database of Bitcoin has more than a billion, so plan for many hours, and for room on the disk for a second copy of the transactions until RocksDB has compacted the first away. It can be interrupted, and starts again from the first transaction, skipping what is done. As before, there is no way back: take a copy first. The parts:

• the migration from version 20 (Haskoin.Store.Database.Migrate);
• the keys that a mempool transaction writes, the summary that is kept for each address they touch, and what the mempool does to the balance and the unspent outputs of an address (Haskoin.Store.Database.Mempool);
• the rules that import blocks and transactions in that layout (Haskoin.Store.Logic.Layered), which the tests hold to the results of the rules of version 20, operation by operation and key by key;
• the queries over both areas (Haskoin.Store.Database.Query.Layered), each on a snapshot of the database, one for each request of the web server, which the tests hold to the answers of the queries of version 20, page by page.

The rules and queries of version 20 (Haskoin.Store.Logic, Haskoin.Store.Database.Query) stay in the library for those tests, and nothing in the store calls them.

Monitoring

• /health returns the result of the latest check as JSON, with status 200 when every check passes and 503 otherwise. The checks are: index lag behind the headers, age of the last block and of the last transaction, transactions waiting to be imported, and connected peers. Point your load balancer at it.

• Without Redis a store that has a cache goes on answering, from its own database, and /health says 203 instead of 200, with the header Haskoin-Degraded: redis and "redis": {"ok": false} in its JSON. A request waits at most two seconds for Redis before the store gives up on it, and the store tries again at every health check. 203 is in the range that load balancers take for healthy, so the store stays in rotation; alert on anything but 200 to hear of it.

• StatsD: with --statsd, the store reports the timing of every kind of request (web.*), database reads (db.*), cache hits, misses, reloads and full reloads (cache.*), and block store gauges such as block and header heights and mempool size (store.*). A server of a fleet reports what it does with the fleet's cache of xpubs under fleet.cache.*:

  Metric	Kind	Meaning
  fleet.cache.hits	counter	Requests about an xpub whose balances the cache gave
  fleet.cache.misses	counter	Requests about an xpub that the database answered
  fleet.cache.builds	counter	Entries that this server built, or added addresses to
  fleet.cache.build	timing	Milliseconds to build an entry
  fleet.cache.abandoned	counter	Builds given up because the fleet or the entry moved on
  fleet.cache.evictions	counter	Entries deleted for being the least recently used
  fleet.cache.emptied	counter	Times that this server emptied the cache
  fleet.cache.xpubs	gauge	Xpubs in the cache, whichever server cached them
  fleet.cache.addresses	gauge	Addresses that those xpubs have between them

  Misses include small xpubs, which are never cached, so the share of hits says how much of the load the cache takes, not how well it holds what it has. fleet.cache.emptied should move only at a reorganisation or when a server stops at the wrong moment: the log of the server says why each time, at the level of information. With --debug it also has a line for each entry built and each block that the cache was brought to.

• /dbstats returns RocksDB memory use per column family and RocksDB's own statistics, as text.

• In a fleet (--fleet), any request can be answered 503 with {"error": "behind-fleet"} by a server that does not have the block that the fleet answers from. Have the load balancer try another server, and alert if one keeps saying it: its /health will say why.

• Memory. The resident memory of the store is its GHC heap plus RocksDB: the block cache and memtables fill up to --block-cache-mb and --write-buffer-mb, then stay there. The executable is built with --disable-delayed-os-memory-return, so memory that the heap no longer needs goes back to the system at once. Without it, Linux keeps counting returned memory as resident (LazyFree in /proc/<pid>/smaps) until it needs it, and the heap looks as if it never shrinks after a peak, such as the full reload of a large cached xpub. Pass +RTS --disable-delayed-os-memory-return -RTS yourself if you build the store some other way. To tell such memory from a leak, see stress/results/2026-11-09-memory/README.md: how to read smaps, and a heap profile of a busy store, with live data flat at about 10 MiB.

• Logs go to standard error, one line per message, tagged with the component, such as [Info#BlockStore] Best block: ....

Using it from Haskell

Query a running server with haskoin-store-data:

import Control.Monad.Except (runExceptT)
import Data.Default (def)
import Haskoin (btc, textToAddr, withContext)
import Haskoin.Store.WebClient

main :: IO ()
main = withContext $ \ctx -> do
  let cfg = def {net = btc} -- https://api.haskoin.com/btc/...
      Just addr = textToAddr btc "bc1q…"
  r <- runExceptT $ apiCall ctx cfg (GetAddrBalance addr)
  print r

The client appends the network name to host, as in the layout of api.haskoin.com. To reach your own server, put it behind a proxy that maps /btc/ (or the name of your network) to it.

Or embed the indexer in your own program with haskoin-store: withStore starts the node, the block store and the cache from a StoreConfig, and the functions of Haskoin.Store.Database.Query.Layered query its database in IO, on a view of it taken with withView, such as withView store.db (\v -> getBalance v addr). Subscribe to Store.pub to receive StoreEvents.

Every module has Haddock documentation; build it with stack haddock.

Packages

Haskoin Store is two Haskell packages, each in its own repository:

• haskoin-store, this repository: the indexer, the web server and the executable.
• haskoin-store-data: the data types, their JSON and binary encodings, and a client for the API (Haskoin.Store.WebClient). Use this one to talk to a Haskoin Store server from Haskell. It is small to depend on, as it has no database or node.

They were released together from this repository up to 1.8.1. A change to the wire format is made in haskoin-store-data, released, and then picked up here; swagger.yaml stays here and describes the server.

Development

stack build --test --no-run-tests
stack test                     # includes a 6000-block heap-growth test
scripts/format                 # ormolu, the code style used here
scripts/bench-web.sh           # load-test the API with wrk
scripts/stress-cache.sh        # stress the Redis cache, with a report
scripts/check-local-deps.sh    # what CI checks: no local paths in stack.yaml
scripts/install-system-libraries.sh  # the C libraries, on Ubuntu or Debian

stack.yaml must pin released versions of every dependency. Pointing extra-deps at a sibling checkout such as ../haskoin-core is fine for local testing, but CI rejects it.

The tests use hspec. The unit tests need no network access. The integration tests (Haskoin.Store.IntegrationSpec) run regtest chains of Bitcoin Core and Bitcoin Cash Node, and Valkey, in Podman containers, and check the store against them: the exact gap, spends and change, mempool order, multiaddr counts, reorganisations, and that the cache answers exactly like the database on every xpub endpoint.

• They are skipped if Podman cannot run containers, or if HASKOIN_STORE_NO_PODMAN is set.
• The first run builds the image localhost/haskoin-store-test, which needs network access: Ubuntu 26.04 pinned by digest, Valkey from a fixed snapshot of the Ubuntu archive, and the release tarballs of both nodes, checked against SHA-256 hashes pinned in test/Haskoin/Store/Integration/Image.hs. No third-party image is used.
• Containers are removed when the tests end, data lives in tmpfs mounts and under /dev/shm, and every container has a memory limit.
• Set HASKOIN_TEST_LOG=1 to print the log of the store under test.

The test suite runs with at most four cores and a 4 GiB heap, so that a runaway test fails instead of exhausting the machine.

package.yaml is the source of the package description. Change it rather than haskoin-store.cabal, which hpack generates.

Stress test of the cache

scripts/stress-cache.sh [OUT_DIR] [-- OPTIONS] measures the cache while it works hard. It runs a regtest chain of Bitcoin Core, Valkey and wrk in Podman containers from the test image, and the haskoin-store binary with the cache on. A few wallets, whose keys the stress program (stress/Main.hs) holds, make hundreds of thousands of transactions between them, signed by the program and sent to the node in batches, with a block after each batch. Every transaction changes two cached xpubs, so the cache reloads them over and over, while wrk asks the server about the xpubs on five endpoints (summary, balances, transactions, unspent outputs and multiaddr) the whole time. At the end it compares the cached answers with those of the database, and scripts/stress-report.py writes OUT_DIR/report.md: how fast the transactions went in, how many reloads the cache did, how many of them read the whole xpub, and how long they took as the xpubs grew, how far the server fell behind, its CPU and memory, the memory of Valkey, the commands that blocked Redis for 10 ms or more (from its slow log, by script, with the number of arguments of each, which grows with the size of a commit), and the latencies of each endpoint over time.

• The defaults are 200,000 transactions between 4 wallets of 200 receiving and 200 change addresses, 2,000 per block; pass -- --help for the options, and -- --txs 20000 for a run of a few minutes. A default run takes about half an hour and up to about 7 GiB of memory.

• The raw measurements stay in OUT_DIR as JSON lines (rounds.jsonl, samples.jsonl, wrk.jsonl), with summary.json and the log of the server.

• No container is left behind, even if the script is interrupted or killed: the program removes its containers when it ends, and the script removes any that carry its label and stops the server.

• Results of past runs are kept in stress/results/<date>/ for comparison: the report, the measurements (without the server log), and a README with the commits, the machine and what the run found. A new run worth keeping goes there too.

• To look for a leak, run the server with a heap profile by type, which needs no profiling build, and summarise it: the live heap over time and the types of closure that grew most.

  scripts/stress-cache.sh /tmp/run -- --txs 60000 \
    --server-rts '-N4 -M2g -T -hT -i10 -po/tmp/heap'
  scripts/heap-profile-summary.py /tmp/heap.hp
  

Measurements for the layered database

docs/layered-database/ has the programs behind the numbers in docs/layered-database.md, with their output. They are not part of the package; each says in its header how to build and run it.

• MigrateBench.hs times the migration to the layered layout against wiping the same mempool.
• QueryCost.hs writes the same synthetic history into a database of each layout, with each set of rules, and times both: importing blocks and mempool transactions, and every kind of query, for addresses and wallets that have nothing, a little and much in the mempool. It checks that both databases give the same answers.
• FleetCost.hs asks one store the same questions alone and as a member of a fleet, with its mempool in a pool in Redis, checks that the answers are the same, and times both. It asks about wallets a third way too, through the fleet's cache of xpubs.

Contributing

Pull requests are welcome.

• Format the code with scripts/format (Ormolu), and keep every line of code shorter than 120 columns.
• Put scripts in files of their own rather than embedding them: Lua for Redis under src/Haskoin/Store/Cache/lua/, shell under scripts/, called from CI rather than written into the workflow.
• Add tests for the behaviour you change.
• Describe the change under a new version in CHANGELOG.md, which follows Keep a Changelog.
• Update swagger.yaml when you change the API.

Related projects

• haskoin-core has the Bitcoin data types and protocol.
• haskoin-node connects to peers and syncs headers.
• nqe, rocksdb-haskell-jprupp, rocksdb-query and statsd-rupp provide concurrency, storage and metrics.

License

Haskoin Store is released under the MIT License.