bdk icon indicating copy to clipboard operation
bdk copied to clipboard

Syncing/Rescanning super issue

Open LLFourn opened this issue 2 years ago • 3 comments

There seems to be quite a few independent efforts to define and address problems with the syncing/scanning API. I thought it would be nice to organize them in one place and have a single PR that does all of them.

  1. Rename "scan" to "rescan": Anything that is using stop_gap style scanning must necessarily be assuming that another wallet must have given out addresses prior to this one. The name rescan communicates this better (h/t @danielabrozzoni). Related #1112.
  2. (#1153) We should have data structures defined in bdk_chain that define bundles of things a Wallet (or any other system) could be interested in. I think these should be:
/// A list of blockchain entities for which we want to receive any related transaction data.
struct SyncRequest {
    /// transactions that spend from or two these script pubkeys
    spks: Vec<ScriptBuf>,
    /// Transactions with these txids 
    txids: Vec<Txid>,
    /// Transactions with these outpoints or spend from these outpoints
    outpoints: Vec<OutPoint>,
    /// The local chain checkpoint. The sync process will return a new chain that extends this one.
    checkpoint: Option<CheckPoint>
}

/// Script pubkeys indexed by their keychain.
struct RescanRequest<K, I> {
    /// Iterators of script pubkeys indexed by the keychain index
    spks_by_keychain: BTreeMap<K,I>,
   /// The local chain checkpoint. The scan process will return a new chain that extends this one.
    checkpoint: Option<CheckPoint>
}
  1. Now the problem with the above is that we want to be able to easy log when a new spk etc is drawn by the blockchain. So really we can't have these fields public -- internally each thing should be Box<dyn Iterator<Item = ...>>. Then when you add a new items to the SyncRequest it should be something like SyncRequest::append_spks(&mut self, spks: impl IntoIterator<Item=ScriptBuf>) which would append the new iterator onto the existing one with chain and replace it.
  2. Then we need APIs in wallet that return these structs (https://github.com/bitcoindevkit/bdk/issues/1195)

LLFourn avatar Nov 06 '23 00:11 LLFourn

We will discuss on this on Tues at the call but your recommendations above all look good to me. I'll start updating #1194 unless anyone has already started working on it or thinks it'd be better to start a new PR.

notmandatory avatar Nov 06 '23 03:11 notmandatory

Given Evan's PR (#1178 ), I believe the checkpoint fields on both structs don't have to be an option anymore.

vladimirfomene avatar Nov 07 '23 12:11 vladimirfomene

Adding this draft PR to the discussion: https://github.com/vladimirfomene/bdk/pull/2

chrisguida avatar Nov 22 '23 16:11 chrisguida

completed with #1403

notmandatory avatar May 10 '24 19:05 notmandatory