matano icon indicating copy to clipboard operation
matano copied to clipboard

Feature Request: Add options pertaining to snapshot expire schedule as part of config

Open rams3sh opened this issue 1 year ago • 2 comments

As discussed in discord, some community members including me have been facing inconsistent timeouts and errors during the snapshot expiry process.

There seems to be some bug with Athena and a case has been raised by me with regards to it. In parallel, to overcome the issue of timeouts, I tried experimenting by changing the schedule and stepfunction timeout to 30m from 1hr and it worked well for me. Expiry averages to 16 mins of running before failing with ICEBERG_VACUUM_MORE_RUNS_NEEDED and then subsequent query sucesssfully executes with average of 30 seconds for my volume of logs with this new setting!!

It would be helpful, if the schedule and stepfunction timeout is kept as part of config so that the consumer can find the sweet spot where the expiry works as expected depending on the size of the logs they ingest. This will also help in managing the athena related issue until it gets resolved.

rams3sh avatar May 19 '23 06:05 rams3sh