trueblocks-core icon indicating copy to clipboard operation
trueblocks-core copied to clipboard

One possible mode of operation

Open tjayrush opened this issue 3 years ago • 0 comments

  1. User creates a file containing address (in exactly the same way he/she does now with chifra monitor).
  2. User runs scraper
  3. When scrape consolidates a block, and any of the addresses are in that chunks, and if the user has it so configured (consider two options --pin and --all as per chifra init):
    • if not --pin and not --all, store only the bloom on disc (remove the index chunk),
    • if not --pin and --all, store both bloom and index chunk on disc,
    • if --pin and not --all, store and pin the bloom,
    • if --pin and --all, store and pin both, Otherwise, store nothing. (or maybe just the bloom).

Upshot: using monitors, we can allows the user to either init or scrape and pin only their own data even on an ongoing basis.

(This is not completely thought out, but it's close...) The idea is that now (as it works now), once the user starts scraping, he/she is taking on the burden of all chunks thereafter, whereas if they had a monitored addresses file, only those chunks that mattered could be stored.

tjayrush avatar Sep 16 '22 02:09 tjayrush