hirosystems
Not possible to import BNS data without first syncing
From conversation with @rafaelcr and related issue: https://github.com/stacks-network/stacks-blockchain-docker/issues/90
With the release of v5.0.0, BNS_IMPORT_DIR is no longer used during starting the API normally.
In previous versions of the API, with this env var defined - the API would process the BNS data and add DB records.
In v5.0.0, i understand this work is being offloaded to the event-replay function. In my testing, i noticed that starting the API normally alongside postgres and a stacks-blockchain node (with STACKS_EXPORT_EVENTS_FILE defined) works as expected and the tsv file is populated with blocks.
Stopping the API after a few blocks, then running the event-replay allows the BNS data to be imported normally.
What doesn't work (and i think this would be the use case for most people) is importing BNS data before running the API normally (i.e. syncing from genesis).
- if you haven't started the API, no tsv file exists and the
import-eventsfunction will fail on file not found (expected) - using an empty tsv file gets past the previous error, and i was expecting the BNS import to proceed but it does not:
{12:07}~/Git/blockstack/public/stacks-blockchain-api:master ✗ ➭ touch file.tsv
node ./lib/index.js import-events \
--file file.tsv --wipe-db --force
{"level":"info","message":"Shutting down... received beforeExit.","timestamp":"2022-09-21T16:07:44.961Z"}
{"level":"info","message":"App shutdown successful.","timestamp":"2022-09-21T16:07:44.963Z"}
tl;dr - it's not possible to import BNS data until you've started the API normally and synced a few blocks. I believe it should be possible via cmd-line switch or natively to import this BNS data before you start syncing the API.
Makes sense.
One of the reasons for this change was to get the genesis block info required for correctly importing BNS, which we can't know if importing from scratch.
We still have the config_state table with some global singleton values in order to track state like this.
Perhaps we could add a new field like bns_import_needs_genesis: boolean, which we could use to first import BNS data with empty genesis block info (setting bns_import_needs_genesis=true), then on genesis block receipt, we can check bns_import_needs_genesis to determine if the "partially" inserted BNS data needs a giant UPDATE.
@rafaelcr thoughts?
That's a good idea @zone117x ... we should aim for a solution that works both for event replay and genesis sync, and this sounds like it could fit the bill.
