cardano-db-sync icon indicating copy to clipboard operation
cardano-db-sync copied to clipboard

Allow to set folder where to download snapshot in postgres

Open APshenkin opened this issue 3 years ago • 6 comments

OS Your OS: DOCKER

Versions The db-sync version (eg cardano-db-sync --version): 12.0.0 PostgreSQL version: 11.5-alpine

Build/Install Method The method you use to build or install cardano-db-sync: Docker

Run method The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none): Docker

Problem Report Currently snapshot downloads to root / directory. So it's not possible to mount the folder where this snapshot should be fetched. So it downloads snapshot to container and if docker is running on machine, where root disk is small, you will not be able to download and extract the data

APshenkin avatar Jan 27 '22 21:01 APshenkin

Same issue and we need a way to specify a folder at least for downloaded snapshot archive.

air3ijai avatar Jun 08 '22 10:06 air3ijai

Any plans to implement this? I don't want my container to be multiple GB's just because of the snapshot file.

It is currently impossible to stay below docker's default container size limit 10 GB

RdeWilde avatar Sep 07 '22 13:09 RdeWilde

As a temporary solution I think you can download the file and put it into any location on your machine. Then mount it to the container root like in this part of docker-compose.yml:

  volumes:
      - /data/db-sync-snapshot-schema-13-block-7791698-x86_64.tgz:/db-sync-snapshot-schema-13-block-7791698-x86_64.tgz
      - /data/tmp:/tmp

Do not forget to mount some additional dir to /tmp because it will start to unpack the archive there

Once the data is loaded you can delete these mounts and the archive

dzmitrykliapkou avatar Oct 26 '22 10:10 dzmitrykliapkou

The snapshot is downloaded in current working directory, so setting the working directory should allow to choose where the download is done. ie., for docker-compose:

 working_dir: /var/lib/cexplorer

or --workdir= for docker.

jbgi avatar Nov 08 '22 09:11 jbgi

Thank you @jbgi @APshenkin does this provide a solution? Can we close?

kderme avatar Nov 09 '22 19:11 kderme

the solution by @jbgi was not complete for me. You need to set the working_dir, but you also need to mount a volume on /tmp

The package is downloaded to working_dir and then extracted into /tmp. It might be useful to have a line or two on this in the snapshot restore readme

mikemorency avatar Apr 27 '23 18:04 mikemorency