Set DATADIR with environment variable

[RobertHosking]

I would like to set some environment variable to configure docker to do the IBD in a certain directory on the host machine.

e.g. export BITCOIN_DATADIR='/path/to/my/big/hdd'

It would also be nice to set the DATADIR for all services. This way I could have a fairly portable installation of BTCPayServer. Migrating to a new host machine would be as simple as mounting the DATADIR directory from a previous installation and running the docker-compose.

I haven't dug into the code yet, but it seems like this would be a simple find and replace of all instances where a volume is mapped replaced with the appropriate environment variable.

Is this functionality desired? Are there any roadblocks to be aware of? I'd like to add this feature if possible.

[maltokyo]

Why not just do the easy way, that is "upgrade proof". Assuming your big hard drive is mounted at /blockchain, shut down the server, then do this, then start it again.

root@lnd:~# mv /var/lib/docker/volumes/generated_bitcoin_datadir /blockchain/generated_bitcoin_datadir
root@lnd:~# ln -s /blockchain/generated_bitcoin_datadir /var/lib/docker/volumes/generated_bitcoin_datadir

[RobertHosking]

@maltokyo symlinks would solve this problem. But IMO, it's not the most elegant solution. I'd like to keep all configuration in the docker-compose instead of creating 5-10 symlinks for every service (bitcoind, lnd, btcpay, db, blockexplorer, webserver, etc).

I'll probably use this method, for now anyways.

[NicolasDorier]

@RobertHosking this is not as easy. Where bitcoin folder is saved is entirely decided by Docker. Some of our docs or script assume it is in /var/lib/docker/volumes/generated_bitcoin_datadir so those would stop working.

I advise against symbolic link, as then some docker volumes command does not work correctly. (Sadly all lunanode deployment use symbolic link)

Instead the best is to make a mount point.

[RobertHosking]

@NicolasDorier correct me if I'm wrong, but currently, the generated docker-compose.yml uses named volumes. This way, docker is in control of what folder is used. E.x.

bitcoind:
    ...
    volumes:
    - "bitcoin_datadir:/data"

If instead, we explicitly mapped the volume from environment variable.

E.x. $ export BITCOIN_DATADIR='/path/to/my/big/hdd'

bitcoind:
    ...
    volumes:
    - "${BITCOIN_DATADIR}:/data"

Of course, this would be done by scripts. So the generated docker-compose.yml would be:

bitcoind:
    ...
    volumes:
    - "/path/to/my/big/hdd:/data"

Wouldn't docker use the provided directory to store data for that volume? I don't understand why this is not an easy feature. Is the problem with sharing data between containers?

[maltokyo]

I guess the issue is what is “/“, root directory is not according to your overall file system, its docker’s “root” dir. so even if you specify the link to /your/big/hdd, it would still look for that inside /var/lib/docker/volumes.

Using “mount --bind” should work.

[NicolasDorier]

@RobertHosking good point. I am annoyed about something though:

1. Our docs mention /var/lib/docker/volumes/generated_bitcoin_datadir
2. Our scripts assume this as well.

And we can't easily rewrite our scripts to use docker volumes instead, because docker it crash if generated_bitcoin_datadir is a symbolic link afaik, but all lunanode deployment use symbolic links.

Nevertheless I agree this can be good feature if the person is aware of the downside.

so even if you specify the link to /your/big/hdd, it would still look for that inside /var/lib/docker/volumes.

This would work as expected. The fact it starts by / means it is not a named volume but a host mapped volume.

@RobertHosking I am trying to think about a way to do that for ANY volume that does not require polluting with many environment variable. I think this is reasonable.

[NicolasDorier]

@RobertHosking can you check something: What do you need exactly to modify? I think the volumes is not enough, you need as well to modify the global volumes section. (you can try by modifying the generated docker-compose and using btcpay-up.sh)

[RobertHosking]

@NicolasDorier

I am trying to think about a way to do that for ANY volume that does not require polluting with many environment variable.

Maybe it's time to use a configuration file? Just a thought...

I think just getting a BITCOIN_DATADIR volume working would be a huge step as it accounts for +90% of the total data stored by BTCPay. I'll do some digging and report back.

[NicolasDorier]

Maybe it's time to use a configuration file? Just a thought...

That's what the docker-fragments are.

[RobertHosking]

I have successfully changed the docker-compose.generated.yml to use an external drive as the bitcoin_datadir.

TL;DR Replace all instances of the named volume with an absolute path to the directory you'd like docker to use.

My docker-compose.generate.yml has a top-level global volumes section

volumes:
  nginx_conf:
  nginx_vhost:
  nginx_html:
  nginx_certs:
  btcpay_datadir:
  tor_datadir:
  tor_torrcdir:
  tor_servicesdir:
  nbxplorer_datadir:
  postgres_datadir:
  bitcoin_datadir:

These are named volumes. See the Docker Compose docs on Volumes.

The named volume bitcoin_datadir is the one I'm interested in. I can see that it's being used in two services:

• bitcoind has volume bitcoin_datadir:/data
• nbxplorer has bitcoin_datadir:/root/.bitcoin

We can replace bitcoin_datadir with an absolute path like /path/to/my/big/hdd. Now the volumes section of bitcoind reads:

  volumes:
    - "/path/to/my/big/hdd:/data"
    - "tor_datadir:/home/tor/.tor"

and nbxplorer

volumes:
    - "nbxplorer_datadir:/datadir"
    - "/path/to/my/big/hdd:/root/.bitcoin"

I left the bitcoin_datadir unchanged in the global volumes section. It works with or without it.

So this got me thinking... is there a way to map a named volume to a host path?

Something like this in the global scope:

# fyi: this will not work
volumes:
  bitcoin_datadir:/path/to/my/big/hdd

Then the rest of the file could remain unchanged. Docker would use the named volume but just map it to the path on the host machine.

According to this blog post, this is possible but it's not intuitive.

I have not tried it but it uses additional params available to named volumes and looks something like this:

volumes:
  bitcoin_datadir:
    driver: local
    driver_opts:
      type: none
      device: /path/to/my/big/hdd
      o: bind

It's very ugly tho.

Also, note that I had to remove the existing generated_bitcoin_datadir volume as well as the btcpayserver_bitcoind container to make the switch to the new folder. Probably could have stopped btcpayserver then copied the data in the generated folder to the new destination before starting like @maltokyo suggested to avoid this.

[RobertHosking]

@NicolasDorier How should this be implemented?

[RobertHosking]

Just confirmed that you can move a fully synced blockchain to another directory and have it work with no issues.

Steps

1. Preform full sync using default bitcoin_datadir volume
2. Stop BTCPay - $ btcpay-down.sh
3. Copy all files from /var/lib/docker/volumes/generated_bitcoin_datadir/_data to destination
4. Replace all instances of bitcoin_datadir in docker-compose.generated.yml to the full path of destination
5. Start BTCPay - $ btcpay-up.sh

[NicolasDorier]

@RobertHosking thanks for looking into this. I will work on a way to replace arbitrary docker volumes via docker fragments by modifying our docker file generator.

[LouisWayne]

Any updates for this?