coolify icon indicating copy to clipboard operation
coolify copied to clipboard

[Bug]: Solution for volumes in compose files is quite "simple", today is very messy

Open renatoaraujoc opened this issue 4 months ago • 0 comments

Error Message and Logs

Hello!

In my current self-hosted system I have many zfs pools where I create many zfs datasets that are specific for each kind of app's volume I'll be working with.

Today, Coolify will try its hardest to simply dynamically prepend the service's uuid to the volume you're trying to create and this makes it impossible to segment my volumes in my system. I've also seen lots of people in these issues complaning that they're having trouble to define volumes as well, some have the need to use NAS Server and its understandable because it's my direction in the future as well.

Today Coolify won't:

  • Respect the externally created volumes
  • Respect pre-defined bind/volume mounts in string form
  • Respect pre-defined bind/volume mounts in object form
  • Respect the docker-compose file's declared volumes

Everything you do, if you're not caught in some 'ifs' in the docker-compose parsing code (I read of all it), it'll just don't care and will attach a dynamic volume at boot time for the app and even bug out the Storage section with many randomly created volumes, it gets very messy.

There are 2 ways I figured this can be solved, one is possibly breaking and the other not so much.

1st way -> If you want to have your volumes created in a dynamic way for a docker-compose stack you're deploying, you'll have to do it like:

volumes:
  my-grafana-data:
    name: my-grafana-data-renamed # optional, needs to be respected
    external: true # optional, needs to be respected

services:
  grafana:
    volumes:
       #_dynamic_ would tell that this volume will handled by Coolify internally (which what is default today)
       - '_dynamic_grafana-data:/var/lib/grafana' #abcd-grafana-data:/var/lib/grafana
       - type: bind
          source: _dynamic_grafana-data # abcd-grafana-data
          target: /var/lib/grafana
       # without _dynamic_, you're responsible to define the volumes, either inside this compose or by
       # defining it with `docker volume create`
       - 'my-grafana-data:/var/lib/grafana'

The above can be super breaking change given how all the services are done today, but hey, we're in beta phase. I believe that this is the best solution.

2st way -> Make Coolify assert that volume xyz exists externally with docker volume ls before trying to do any dynamic volume substitution OR respect the volumes: section.

  1. If volume xyz exists externally, don't touch its references in the services, else make it dynamic.
  2. Make Coolify respect the manual definition of a volume like below:
volumes:
  # you're telling coolify that you're handling this volume
  # thus it should not make it dynamic and neither make any substitution with it because we're handling it
  # reference for name: https://docs.docker.com/reference/compose-file/volumes/#name
  # reference for external: https://docs.docker.com/reference/compose-file/volumes/#external
  my-grafana-data:
    name: my-grafana-data-renamed # needs to respect this
    external: true # needs to respect this

The way above would be non-breakable with the current way Coolify works today.

Again, in my point of view here, if I want Coolify to make stuff dynamic for volumes, I'd like to make it clear in the docker-compose file with something like _dynamic_{{volumeName}}:/path_on_container, else me (external: true) or docker-compose (external: false) are responsible to handle it.

Please, we have to settle down these issues with volumes.

Thanks,

Renato

Steps to Reproduce

None.

Example Repository URL

No response

Coolify Version

v4.0.0-beta.360

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Ubuntu 24.04

Additional Information

No response

renatoaraujoc avatar Oct 19 '24 20:10 renatoaraujoc