ethereum-etl icon indicating copy to clipboard operation
ethereum-etl copied to clipboard

Add option to disable "last_synced_block.txt"

Open nickcorin opened this issue 2 years ago • 2 comments

When using the stream command, you are required to either provide only of either the --start-block flag or the --last-synced-block-file flag. However, even when using --start-block, the last_synced_block.txt file is created preventing you from using the --start-block flag again until you remove the file which as been created.

In my use case, I would prefer to provide the --start-block flag each time that I start streaming. This doesn't seem possible without cleaning up the file each time the process is interrupted.

A normal use case for this would be if a consumer of the data is keeping their own cursor. You would want to provide your cursor to the ethereumetl in the --start-block flag rather than it defaulting to using the file.

I see two ways of implementing this:

  1. Adding a --disable-last-synced-block-file flag which would prevent the file from being created.
  2. Removing the functionality which defaults to using last_synced_block.txt, and only creating the file when the --last-synced-block-file flag is provided.

I'm happy to submit a PR for either of these solutions.

nickcorin avatar Mar 08 '22 13:03 nickcorin

Thanks for the feedback! This is indeed sometimes annoying.

Option 2 seems simpler to me however it wouldn't be backward compatible, unless we invent some special string indicating disabling this file like /dev/null but this is too cryptic. So I'd prefer to go with option 1.

medvedev1088 avatar Mar 30 '22 15:03 medvedev1088

@medvedev1088 how about we just set the value in last_synced_block.tx to start block if the --start-block flag present ?

sfsf9797 avatar Sep 24 '22 13:09 sfsf9797