Update bdk_bitcoind_rpc module to work with a pruned bitcoind node

[notmandatory]

Summer of Bitcoin Project Proposal

Description

The current bdk_bitcoind_rpc module requires full access to transaction history using a non-pruned node (a node storing the full blockchain, which at the time of writing is over 400GB).

However, even without a wallet's full transaction history it is still possible to compute the wallet balance and be able to spend with only a pruned bitcoind node using the RPC method scantxoutset. This project will update the existing bdk_bitcoind_rpc module to work with a pruned bitcoind node.

Expected Outcomes

• Updatefull_scan to use the scantxoutset if the bitcoind node is pruned.
• Create tests to confirm full_scan works with a pruned node.
• Update rpc examples and test with a pruned bitcoind node.

Resources

• bitcoin core node setup.
• bitcoin rpc commands.
• issue 895.
• https://developer.bitcoin.org/reference/rpc/scantxoutset.html
• Add bitcoind_rpc chain source module. #1041
• Introduce block-by-block API to bdk::Wallet and add RPC wallet example #1172

Skills Required

• Experience with git. Guide
• Experience with rust. First seven chapters of the book
• Able to setup a local core node in regtest and testnet mode.
• Familiarity with bitcoind RPC commands.

Mentor(s)
@notmandatory

Difficulty
Hard

Competency Test

• Install rust, compile and run all bdk examples and tests.
• Setup a local Bitcoin Core pruned node daemon in regtest mode.
• Make a wallet with example_bitcoind_rpc_polling example wallet and receive and send regtest bitcoin.

[rustaceanrob]

Mind if I work on this? I'm not SOB but I am doing a similar program haha

[c0llinx]

Is this still open? Would like to give it a shot

[LLFourn]

Not sure if our architecture actually supports this kind of usage. The problem is how do you figure out when an output has been spent? The only way is that it is no longer returned from the scantxoutset output. But in bdk_chain we tell it outputs are spent by providing a full transaction that spent it. We can't get that here.

It can be used to find your current balance, but can't be used in a way where you persist anything. So the workflow would be to just scantxout then get the outputs, create a transaction from them and spend them. You could .insert_txout them into a temporary Wallet and create transactions from there. I think that's the only way to get value from bdk with this atm.

[star-gazer111]

Hey @notmandatory the link you shared for rpc commands (https://developer.bitcoin.org/reference/rpc/) is somehow not working.

[star-gazer111]

Hey @notmandatory the link you shared for rpc commands (https://developer.bitcoin.org/reference/rpc/) is somehow not working.

I think this can be a good alternative resource for the same.

[jaoleal]

@star-gazer111 https://chainquery.com/bitcoin-cli i prefer to use this one, its kinda better.

[notmandatory]

@LLFourn what if we use the RPC syncing as it is now but with a bitcoind node set to "manual" pruning? then we just have to add functionality to call the "pruneblockchain" command after syncing to remove old (maybe older than 1000 deep or so) already ingested blocks.

[LLFourn]

@notmandatory yeah cool I didn't know about this setting but that also works. I feel like the usage is a little niche:

1. You don't want to run an archival node
2. You can guarantee your wallet will come online often enough to drain the blocks and prune.
3. You only have one wallet system.

Certainly not an unlikely situation within a small Bitcoin startup. Also can exist in user facing software that actually embeds a pruned node.

[luisschwab]

This looks interesting, I think I will give it a shot