mempool-dumpster icon indicating copy to clipboard operation
mempool-dumpster copied to clipboard

Dockerize production setup

Open metachris opened this issue 1 year ago • 0 comments

Goal:

  • Run mempool dumpster instances in Docker
  • Uploading hourly files to R2/S3 (with a bit of delay)
  • Nightly processing should pull all these files from S3, and process them from there

Notes:

  • collector and processor need access to reliable EL nodes, for checking of inclusion status
  • nightly processing is using a lot of memory to process all the transactions (up to ~6.5GB)

Links:

  • https://github.com/aws/aws-sdk-go
  • https://developers.cloudflare.com/r2/examples/aws/aws-sdk-go/

metachris avatar Jun 24 '24 07:06 metachris