Restart of Colima resets all docker containers...?

[codethought]

Description

In trying to move from Docker Desktop to Colima I've run into a problem that I cannot figure out how to resolve. I'm able to start Colima and create docker containers via docker-compose without issue. However if I restart colima it is as I'm doing a new install/reconfiguration of the container that I've already had running. Shouldn't the state of the containers be persisting during restarts the way they did when I used Docker Desktop?

Version

Colima Version: 0.5.2 Lima Version: 0.14.2 Qemu Version: 7.2.0

Operating System

• [X] macOS Intel <= 12 (Monterrey)
• [ ] macOS Intel >= 13 (Ventura)
• [ ] macOS M1 <= 12 (Monterrey)
• [ ] macOS M1 >= 13 (Ventura)
• [ ] Linux

Output of colima status

INFO[0000] colima is running using QEMU INFO[0000] arch: x86_64 INFO[0000] runtime: docker INFO[0000] mountType: sshfs INFO[0000] socket: unix:///Users/codethought/.colima/default/docker.sock INFO[0000] kubernetes: enabled

Reproduction Steps

1. Start coliima 2.create a container with docker-compose : for example

 version: '3'
 services:
     portainer:
         container_name: Portainer
         image: portainer/portainer-ce:latest
         restart: always
         ports:
         - "9000:9000"
         volumes:
         - /var/run/docker.sock:/var/run/docker.sock
         - /Volumes/Data/src/repos/docker/portainer/data:/data

3. Go to portainer website at localhost:9000 and configure a new user.
4. restart colima with colima stop and colima start
5. Go to portainer website You have to reconfigure your user.

Expected behaviour

I should be able to use portainer (or any other app) after a restart of colima without having to completely reconfigure from scratch. Just like I would do in Docker Desktop

here is a copy of my colima.yaml file too

# Number of CPUs to be allocated to the virtual machine.
# Default: 2
cpu: 4

# Size of the disk in GiB to be allocated to the virtual machine.
# NOTE: changing this has no effect after the virtual machine has been created.
# Default: 60
disk: 100

# Size of the memory in GiB to be allocated to the virtual machine.
# Default: 2
memory: 12

# Architecture of the virtual machine (x86_64, aarch64, host).
# Default: host
arch: x86_64

# Container runtime to be used (docker, containerd).
# Default: docker
runtime: docker

# Kubernetes configuration for the virtual machine.
kubernetes:
  # Enable kubernetes.
  # Default: false
  enabled: true
  
  # Kubernetes version to use.
  # This needs to exactly match a k3s version https://github.com/k3s-io/k3s/releases
  # Default: latest stable release
  version: v1.25.4+k3s1
  
  # Disable k3s features [coredns servicelb traefik local-storage metrics-server].
  # All features are enabled unless disabled.
  #
  # EXAMPLE - disable traefik and metrics-server
  # disable: [traefik, metrics-server]
  #
  # Default: [traefik]
  disable:
    - traefik

# Auto-activate on the Host for client access.
# Setting to true does the following on startup
#  - sets as active Docker context (for Docker runtime).
#  - sets as active Kubernetes context (if Kubernetes is enabled).
# Default: true
autoActivate: true

# Network configurations for the virtual machine.
network:
  # Assign reachable IP address to the virtual machine.
  # NOTE: this is currently macOS only and ignored on Linux.
  # Default: false
  address: false
  
  # Custom DNS resolvers for the virtual machine.
  #
  # EXAMPLE
  # dns: [8.8.8.8, 1.1.1.1]
  #
  # Default: []
  dns: []
  
  # DNS hostnames to resolve to custom targets using the internal resolver.
  # This setting has no effect if a custom DNS resolver list is supplied above.
  # It does not configure the /etc/hosts files of any machine or container.
  # The value can be an IP address or another host.
  #
  # EXAMPLE
  # dnsHosts:
  #   example.com: 1.2.3.4
  dnsHosts: {}
  
  # Network driver to use (slirp, gvproxy), (requires vmType `qemu`)
  #   - slirp is the default user mode networking provided by Qemu
  #   - gvproxy is an alternative to VPNKit based on gVisor https://github.com/containers/gvisor-tap-vsock
  # Default: gvproxy
  driver: gvproxy

# ===================================================================== #
# ADVANCED CONFIGURATION
# ===================================================================== #

# Forward the host's SSH agent to the virtual machine.
# Default: false
forwardAgent: false

# Docker daemon configuration that maps directly to daemon.json.
# https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file.
# NOTE: some settings may affect Colima's ability to start docker. e.g. `hosts`.
#
# EXAMPLE - disable buildkit
# docker:
#   features:
#     buildkit: false
#
# EXAMPLE - add insecure registries
# docker:
#   insecure-registries:
#     - myregistry.com:5000
#     - host.docker.internal:5000
#
# Colima default behaviour: buildkit enabled
# Default: {}
docker: {}

# Virtual Machine type (qemu, vz)
# NOTE: this is macOS 13 only. For Linux and macOS <13.0, qemu is always used.
#
# vz is macOS virtualization framework and requires macOS 13
#
# Default: qemu
vmType: ""

# Volume mount driver for the virtual machine (virtiofs, 9p, sshfs).
#
# virtiofs is limited to macOS and vmType `vz`. It is the fastest of the options.
#
# 9p is the recommended and the most stable option for vmType `qemu`.
#
# sshfs is faster than 9p but the least reliable of the options (when there are lots
# of concurrent reads or writes).
#
# Default: virtiofs (for vz), sshfs (for qemu)
mountType: sshfs

# The CPU type for the virtual machine (requires vmType `qemu`).
# Options available for host emulation can be checked with: `qemu-system-$(arch) -cpu help`.
# Instructions are also supported by appending to the cpu type e.g. "qemu64,+ssse3".
# Default: host
cpuType: ""

# For a more general purpose virtual machine, Ubuntu container is optionally provided
# as a layer on the virtual machine.
# The underlying virtual machine is still accessible via `colima ssh --layer=false` or running `colima` in
# the Ubuntu session.
#
# Default: false
layer: false

# Custom provision scripts for the virtual machine.
# Provisioning scripts are executed on startup and therefore needs to be idempotent.
#
# EXAMPLE - script exected as root
# provision:
#   - mode: system
#     script: apk add htop vim
#
# EXAMPLE - script exected as user
# provision:
#   - mode: user
#     script: |
#       [ -f ~/.provision ] && exit 0;
#       echo provisioning as $USER...
#       touch ~/.provision
#
# Default: []
provision: []

# Modify ~/.ssh/config automatically to include a SSH config for the virtual machine.
# SSH config will still be generated in ~/.colima/ssh_config regardless.
# Default: true
sshConfig: true

# Configure volume mounts for the virtual machine.
# Colima mounts user's home directory by default to provide a familiar
# user experience.
#
# EXAMPLE
# mounts:
#   - location: ~/secrets
#     writable: false
#   - location: ~/projects
#     writable: true
#
# Colima default behaviour: $HOME and /tmp/colima are mounted as writable.
# Default: []
mounts: []

# Environment variables for the virtual machine.
#
# EXAMPLE
# env:
#   KEY: value
#   ANOTHER_KEY: another value
#
# Default: {}
env: {}

Additional context

No response

[abiosoft]

This is the issue

 - /Volumes/Data/src/repos/docker/portainer/data:/data

Colima only mounts the $HOME directory by default, so your data is not being preserved.

If you must use a directory outside of $HOME, you need to override the default mounts.

colima start --mount $HOME:w --mount /Volumes/Data:w

[codethought]

@abiosoft tried that before.. when I do I get the following..

[1 error occurred:
        * failed to prepare reverse sshfs for "/Volumes/Mac CanPro Data" on "/Volumes/Mac CanPro Data": failed to mkdir "/Volumes/Mac CanPro Data" (remote): "mkdir: can't create directory '/Volumes/Mac': Permission denied\n": exit status 1

I've tried a couple of derivatives of the command...

colima start --mount $HOME:w --mount "/Volumes/Mac CanPro Data/":w

and

colima start --mount $HOME:w --mount /Volumes/Mac\ CanPro\ Data/:w

Trying to modify the yaml configuration to put the volume in quotes doesn't work either.. The quotes get stripped and I see the same error..

[codethought]

@abiosoft - looks like as a workaround if I use mounts or directories with no spaces in their names I can get things working and persisting between startup and shutdown of colima.. might be work adding to the readme not to use directories with spaces in colima (at least until the issue with that is resolved)..

Thank you for the help.. great work btw..

[abiosoft]

@codethought interesting, I'll try to reproduce that on my end.

If that's indeed the case then it's a bug that should be fixed.

[JackieTreeh0rn]

I'm experiencing this issue as well with 0.6.8 but I'm not even mounting any volumes, just the standard $HOME. I'm only losing containers upon restart not images...

[edit] I take that back. containers were just stopped - docker -ps showed nothing but docker -ps a shows they are still there just stopped. Adding restart: always to my compose ymls took care of it. I don't recall having to do this before but, that's that...

[nothing2obvi]

I'm having the same issue. If I mount the directories in colima.yaml it doesn't make a difference. I don't have any spaces in these paths.

mounts:
  - location: /Users/username
    writable: true
  - location: /Volumes/External-HDD
    writable: true

[tantweiler]

This is the issue

 - /Volumes/Data/src/repos/docker/portainer/data:/data

Colima only mounts the $HOME directory by default, so your data is not being preserved.

If you must use a directory outside of $HOME, you need to override the default mounts.

colima start --mount $HOME:w --mount /Volumes/Data:w

@abiosoft I only use my $HOME directory for volume mounts but I am facing the same issue that the guys explained here already. Here is my docker compose file that I run with nerdctl compose:

services:
  timotest:
    image: postgres:latest
    container_name: test
    environment:
      POSTGRES_PASSWORD: mysecretpw
      PGDATA: /var/lib/postgresql/data/pgdata
    volumes:
      - /Users/timo/postgres/test:/var/lib/postgresql/data
    ports:
      - "5432:5432"

So if $HOME is meant to be mounted automatically but the container is not restarted is this a bug in colima? I am also not able to restart the container manually. I must delete the old container and re-run nerdctl compose up -d.

[tantweiler]

Short update! It seems to be an issues with containerd runtime. If I choose docker as runtime it works perfectly.