Add a new environment variable that sets a block height until which the API node stays in "Initial Block Download(IBD)" mode

[mutharasus]

Is your feature request related to a problem? Please describe. Currently the API Node / Database does not have a way to go into a "Initial Block Download(IBD)" mode when the node is far behind its other peers. There is a proposed feature that is a likely long term solution. This feature request proposes as a stop gap solution until a permanent fix is implemented on the blockchain itself.

Describe the solution you'd like The solution being proposed is to introduce a new environment variable which when set to a particular block height informs the API Node to go into a "Initial Block Download(IBD)" mode similar to Bitcoin where in it can avoid any processing that could be considered redundant until the node is fully synced up to its peers. Some examples for processing that can be avoided are: REFRESH MATERIALIZED VIEW SQLs that are extremely CPU intensive on the PG instance, Mempool messages, etc.,

The name of the environment variable can be "IBD_MODE_UNTIL_BLOCK=12345" or similar.

Describe alternatives you've considered Other alternatives considered beyond the long term solution mentioned in the problem statement above is, introduce a environment variable which is a boolean flag that can be manually set to indicate the node should operate in a "Initial Block Download(IBD)" mode. The down side with this approach is that the node operator will have to stop the node and restart it with the flag reset once the node is in sync with its peers.

Additional context None

[rafaelcr]

Related to #698

[saralab]

Matt: To Do @BLuEScioN :

• Benchmarks for syncing from Genesis
• Post v6
• Test this in new Dev env

[zone117x]

Testing is blocked on https://github.com/stacks-network/stacks-blockchain/issues/3456

[wileyj]

Testing is blocked on stacks-network/stacks-blockchain#3456

Is this still blocked? it seems unrelated (only evidence we could find is that the burnchain node was just overwhelmed at the time of the errors).

another idea i had when speaking to @mutharasus was rather than adding this as a one-off env var, it might be a cool idea to add something like an API key env var locally that can be used to POST admin changes to the API.

something like this, where the sample endpoint is firewalled for localhost requests:

curl --request POST \
  'http://localhost:3999/extended/v1/admin/setvars' \
  --header 'Content-Type: application/json' \
  --header 'api-key: some-api-key' \
  --data-raw '{
    "IBD_MODE_UNTIL_BLOCK": 12345
  }'

i can think of several other potential operations here as well

[muthusleuths]

I think there is one more thing that is not working as expected @rafaelcr @BLuEScioN.

So when I run the node with the flag IBD_MODE_UNTIL_BLOCK in the config the API node stays in IBD Mode even beyond the specified block number. I also attempted to reboot and still produces the same results.

To get the node off of IBD mode you have to remove the flag completely from the Config and then reboot the API node (including the blockchain node).

Please let me know if I need to create a new issue for this and we should close this one as completed.

Thanks

[rafaelcr]

thanks @muthudotbtc , we can track it in this same issue, let us investigate this and we'll get back to you

[muthusleuths]

Testing is blocked on stacks-network/stacks-blockchain#3456

Is this still blocked? it seems unrelated (only evidence we could find is that the burnchain node was just overwhelmed at the time of the errors).

another idea i had when speaking to @mutharasus was rather than adding this as a one-off env var, it might be a cool idea to add something like an API key env var locally that can be used to POST admin changes to the API.

something like this, where the sample endpoint is firewalled for localhost requests:

curl --request POST \
  'http://localhost:3999/extended/v1/admin/setvars' \
  --header 'Content-Type: application/json' \
  --header 'api-key: some-api-key' \
  --data-raw '{
    "IBD_MODE_UNTIL_BLOCK": 12345
  }'

i can think of several other potential operations here as well

Should this be created as a new issue @rafaelcr / @wileyj ?

[rafaelcr]

@muthudotbtc This is now fixed in v7.1.5 ✅