[Bug]: Solution for volumes in compose files is quite "simple", today is very messy

[renatoaraujoc]

Error Message and Logs

Hello!

In my current self-hosted system I have many zfs pools where I create many zfs datasets that are specific for each kind of app's volume I'll be working with.

Today, Coolify will try its hardest to simply dynamically prepend the service's uuid to the volume you're trying to create and this makes it impossible to segment my volumes in my system. I've also seen lots of people in these issues complaning that they're having trouble to define volumes as well, some have the need to use NAS Server and its understandable because it's my direction in the future as well.

Today Coolify won't:

• Respect the externally created volumes
• Respect pre-defined bind/volume mounts in string form
• Respect pre-defined bind/volume mounts in object form
• Respect the docker-compose file's declared volumes

Everything you do, if you're not caught in some 'ifs' in the docker-compose parsing code (I read of all it), it'll just don't care and will attach a dynamic volume at boot time for the app and even bug out the Storage section with many randomly created volumes, it gets very messy.

There are 2 ways I figured this can be solved, one is possibly breaking and the other not so much.

1st way -> If you want to have your volumes created in a dynamic way for a docker-compose stack you're deploying, you'll have to do it like:

volumes:
  my-grafana-data:
    name: my-grafana-data-renamed # optional, needs to be respected
    external: true # optional, needs to be respected

services:
  grafana:
    volumes:
       #_dynamic_ would tell that this volume will handled by Coolify internally (which what is default today)
       - '_dynamic_grafana-data:/var/lib/grafana' #abcd-grafana-data:/var/lib/grafana
       - type: bind
          source: _dynamic_grafana-data # abcd-grafana-data
          target: /var/lib/grafana
       # without _dynamic_, you're responsible to define the volumes, either inside this compose or by
       # defining it with `docker volume create`
       - 'my-grafana-data:/var/lib/grafana'

The above can be super breaking change given how all the services are done today, but hey, we're in beta phase. I believe that this is the best solution.

2st way -> Make Coolify assert that volume xyz exists externally with docker volume ls before trying to do any dynamic volume substitution OR respect the volumes: section.

1. If volume xyz exists externally, don't touch its references in the services, else make it dynamic.
2. Make Coolify respect the manual definition of a volume like below:

volumes:
  # you're telling coolify that you're handling this volume
  # thus it should not make it dynamic and neither make any substitution with it because we're handling it
  # reference for name: https://docs.docker.com/reference/compose-file/volumes/#name
  # reference for external: https://docs.docker.com/reference/compose-file/volumes/#external
  my-grafana-data:
    name: my-grafana-data-renamed # needs to respect this
    external: true # needs to respect this

The way above would be non-breakable with the current way Coolify works today.

Again, in my point of view here, if I want Coolify to make stuff dynamic for volumes, I'd like to make it clear in the docker-compose file with something like _dynamic_{{volumeName}}:/path_on_container, else me (external: true) or docker-compose (external: false) are responsible to handle it.

Please, we have to settle down these issues with volumes.

Thanks,

Renato

Steps to Reproduce

None.

Example Repository URL

No response

Coolify Version

v4.0.0-beta.360

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Ubuntu 24.04

Additional Information

No response

[crimsonvspurple]

I have a bind volume that is being changed by coolify. Can we get a option to skip the auto volume name management?

[9812598]

I have same problem. Unfortunately, your suggestion didn't help and he still renames and changes volumes.

[anvidev]

I had a similar issue on Coolify version v4.0.0-beta.409. I solved it by checking the "Preserve Repository During Deployment" checkbox on the configuration > general tab under the build section. I'm not entirely sure if this is a hack or the intended way to fix the issue with volumes/mounts for docker compose, but it might be worth a shot for you guys to try.

[notflip]

I had a similar issue on Coolify version v4.0.0-beta.409. I solved it by checking the "Preserve Repository During Deployment" checkbox on the configuration > general tab under the build section. I'm not entirely sure if this is a hack or the intended way to fix the issue with volumes/mounts for docker compose, but it might be worth a shot for you guys to try.

@anvidev Do you define your own volumes in docker-compose with this flag turned on? I have my own volume defintions in docker-compose, without the flag, and the db is being deleted every deploy.

[9812598]

Conig that works for me

version: "3.8"

services: db: image: mysql:5.7 container_name: wp_db volumes: - type: bind source: ./db-init target: /docker-entrypoint-initdb.d - type: bind source: ./db-data target: /var/lib/mysql restart: always environment: MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD} MYSQL_DATABASE: ${MYSQL_DATABASE} MYSQL_USER: ${MYSQL_USER} MYSQL_PASSWORD: ${MYSQL_PASSWORD} networks: - wpsite

phpmyadmin: image: phpmyadmin/phpmyadmin container_name: wp_phpmyadmin depends_on: - db restart: always # ports: # - "8081:80" environment: PMA_HOST: db MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD} networks: - wpsite

wordpress: image: wordpress:latest container_name: wp_site depends_on: - db # ports: # - "8002:80" restart: always volumes: - type: bind source: ./wordpress target: /var/www/html

environment:
  # Database connection
  WORDPRESS_DB_HOST: db:3306
  WORDPRESS_DB_NAME: ${MYSQL_DATABASE}
  WORDPRESS_DB_USER: ${MYSQL_USER}
  WORDPRESS_DB_PASSWORD: ${MYSQL_PASSWORD}
  # Additional PHP environment variables
  MYSQL_DATABASE: ${MYSQL_DATABASE}
  MYSQL_USER: ${MYSQL_USER}
  MYSQL_PASSWORD: ${MYSQL_PASSWORD}

command: sh -c "chown -R www-data:www-data /var/www/html && docker-entrypoint.sh apache2-foreground"
networks:
  - wpsite

volumes: db_data:

networks: wpsite:

But to be honestly, it was a very bad idea overall. WP sites don't need to be uploaded via git

[anvidev]

I had a similar issue on Coolify version v4.0.0-beta.409. I solved it by checking the "Preserve Repository During Deployment" checkbox on the configuration > general tab under the build section. I'm not entirely sure if this is a hack or the intended way to fix the issue with volumes/mounts for docker compose, but it might be worth a shot for you guys to try.

@anvidev Do you define your own volumes in docker-compose with this flag turned on? I have my own volume defintions in docker-compose, without the flag, and the db is being deleted every deploy.

@notflip Yes, I define my own volumes. See my docker compose file below (migrate service). I tested it also with the Coolify's extra options for storage, which worked also. documentation

services:
  db:
    image: postgres:17.2
    container_name: blaekward-db
    restart: unless-stopped
    shm_size: 128mb
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    ports:
      - 5432:5432
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 5s
      timeout: 5s
      retries: 5

  # adminer:
  #   image: adminer
  #   restart: unless-stopped
  #   ports:
  #     - 8080:8080

  migrate:
      image: migrate/migrate
      depends_on:
        db:
          condition: service_healthy
      volumes:
        - ./cmd/migrate/migrations:/migrations
      command:
        [ "-path", "/migrations", "-database", "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}?sslmode=disable", "up" ]

  api:
    build: .
    container_name: blaekward-api
    restart: unless-stopped
    ports:
      - "9090:9090"
    environment:
      POSTGRES_ADDR: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}?sslmode=disable
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "curl --fail http://localhost:9090/ping || exit 1"]
      interval: 30s
      timeout: 5s
      retries: 3

volumes:
  db-data:

[notflip]

@anvidev Something odd happens for me, I define the volumes in my docker-compose.yml like this

    volumes:
      - uploads_data:/app/web/uploads # Alleen de uploads persistent maken

volumes:
  db_data:
    external: true
  uploads_data:
    external: true

And then I see that Coolify transforms it into this (in the docker-compose file that Coolify shows underneath)

volumes:
  db_data:
    external: true
  uploads_data:
    external: true
  to4s4wok88ockwckk88coo8o_db-data:
    name: to4s4wok88ockwckk88coo8o_db-data
  to4s4wok88ockwckk88coo8o_uploads-data:
    name: to4s4wok88ockwckk88coo8o_uploads-data

Maybe I should omit external from the defintion, I'm not sure.

[9812598]

Try to use this

volumes:

• type: bind source: ./db-init target: /docker-entrypoint-initdb.d

Change db-init and docker-entrypoint-initdb.d to your folders

1. Check that your analog of db-init exist in root
2. Check that docker-entrypoint-initdb.d is working (you can do it from docker console)

[notflip]

What's inside db-init and docker-entrypoint.initdb.d?

Try to use this

volumes:

• type: bind source: ./db-init target: /docker-entrypoint-initdb.d

Change db-init and docker-entrypoint-initdb.d to your folders

1. Check that your analog of db-init exist in root
2. Check that docker-entrypoint-initdb.d is working (you can do it from docker console)

What's inside these 2 files then?

[9812598]

in my case is dump.sql as i use it for WP in your case it should be something like this.

volumes: type: bind source: ./uploads_data target: /app/web/uploads

If you use this volumes: - uploads_data:/app/web/uploads collify will rename them

But i'm not shure that you need /app. I just show example. You should ckeck where is your files located and where you want to put them in container

[leopold04]

I'm having a similar issue where mounting volumes to containers simply does not work. How far back would I need to downgrade in order to have volumes working? Would it be a bad idea to downgrade?

[plcdnl]

@anvidev Something odd happens for me, I define the volumes in my docker-compose.yml like this

    volumes:
      - uploads_data:/app/web/uploads # Alleen de uploads persistent maken

volumes:
  db_data:
    external: true
  uploads_data:
    external: true

And then I see that Coolify transforms it into this (in the docker-compose file that Coolify shows underneath)

volumes:
  db_data:
    external: true
  uploads_data:
    external: true
  to4s4wok88ockwckk88coo8o_db-data:
    name: to4s4wok88ockwckk88coo8o_db-data
  to4s4wok88ockwckk88coo8o_uploads-data:
    name: to4s4wok88ockwckk88coo8o_uploads-data

Maybe I should omit external from the defintion, I'm not sure.

hi! i had the same problem. How did you solve?

[9812598]

Honestly, I realized that running WordPress through Docker is a bad idea. The main point of WordPress is that you can edit it from the admin panel, but with Docker, you can do it only via git.