data persistence is missing in docs

[max-tet]

Hi. I was wondering how to mount a directory such that all relevant data is persisted outside of the container itself. The docker commands in the readme don't do that. Would be great if you could add that information. Otherwise, all data would be lost if the container is removed. Thanks!

[JulianPrieber]

This is not something we really support or is something we test.

You can mount a directory with: --volume /docker/data/www:/htdocs \

Again, not something we test or officially support. I'm not sure if this will work without modifying the dockerfile.

Hope this helps 👍

[max-tet]

Thanks for the hint. I tried to get it to work, but unsuccessfully. Mounting /htdocs is too broad, because the whole directory then is empty and the web interface just shows an empty directory listing from nginx.

I tried to change the location of the database with -e DB_DATABASE=/data/db.sqlite and mount /data but it seems like the database must be initialized. If the file does not exist, the application does not work.

I would reccomend to create the database file on application startup if it does not exist yet and apply any migrations that might be needed.

Also, are there other directories that should be mounted? Can a user upload files that are stored somewhere on the filesystem?

In general, when publishing a docker-based application that is not explicitely meant to be stateless, it is really best practice to offer a method to persist data. Usually documenting which directories should be mounted is sufficient.

[JulianPrieber]

Our docker image is just an Apache web server running the web version of LittleLink Custom. For that reason, our two versions don't differ for the simple reason of keeping everything manageable to maintain.

Everything is stored in the htdocs folder in the docker container, including the database.

When creating the docker, I essentially move everything from here into the htdocs folder and then do some permission management for the web server.

COPY littlelink-custom /htdocs
RUN chown -R apache:apache /htdocs
RUN find /htdocs -type d -print0 | xargs -0 chmod 0755
RUN find /htdocs -type f -print0 | xargs -0 chmod 0644 

If you want storage independent of the docker, you'd have to move files into this folder again and probably have to redo the file permission, then everything should work as normal.

[max-tet]

Why don't you build the docker image directly from the littlelink-custom repo? Just put the dockerfile there. You wouldn't have the problem of maintaining two versions and could have a pipeline that builds a new image with every release.

[JulianPrieber]

Why don't you build the docker image directly from the littlelink-custom repo? Just put the dockerfile there. You wouldn't have the problem of maintaining two versions and could have a pipeline that builds a new image with every release.

We do build the docker image from the littlelink-custom repo. Everything is automated when I release a new version. It is actually easier to have the two versions separate. This repository only contains docker files, our main repository contains the application itself.

[max-tet]

I'm not sure I understand why that is easier, but as long as it works for you that's great.

But still, I think for a docker release, it is really important that there is an easy way for data persistence that is well documented. Noone wants their data to be lost when the container is removed. A missing piece in an otherwise very nice project.

[bonelifer]

I agree. Just reorganizing my instance of dockers. And didn't realize there wasn't persistence on this container. Lost everything. :(

[JulianPrieber]

I think I worked out something that works and is easy to set up:

1. Create volume: docker volume create llc

2. Running the Docker image normally while mounting the volume: docker run --detach --name littlelink-custom --publish 80:80 --publish 443:443 --restart unless-stopped --mount source=llc,target=/htdocs julianprieber/littlelink-custom

Something like this

This should move the important files to the volume for persistent data.

If you could please check that this is performing as expected, @max-tet, I'll add it to the documentation.

[max-tet]

Thanks for your effort. It is only half a solution though, sorry. When inspecting the volume llc, you can see that basically the whole application is copied there, not just the user data. You can use e.g. docker run -v llc:/srv -it --rm -p 80:80 filebrowser/filebrowser to have a look.

This will most probably become a problem when upgrading the app. Since lots of parts of the app itself are now persisted in the volume, they will not be upgraded when a new docker image is used.

The correct way to implement data persistence would be to:

1. change the paths of all user data files (sqlite db, uploaded media, etc.) such they are inside their own directory, e.g. /data and not mixed between application files.
2. Make sure that the app starts correctly if the data dir is initially empty. I.e. create and initialize the database if it does not exist.
3. Mount a dir or volume only for the user data directory.

[JulianPrieber]

Thanks for the quick response and help on this! I will look into the method you described and get a third opinion on this.

Granted, this is probably not the best solution.

Copying the whole application was actually intentional. As it is, right now this is the easiest to implement solution I can think of for the user. Copying every directory and file that contains user data that should be persisted is much easier this way. If I'd copy individual folders, I would have to future prove the thing, so it would still include anything that might get added later on.

You might be right on the upgrade part, I will do some testing and see what works best.

[LeoColman]

I don't think there's an issue with copying the whole application.

We can have multiple volumes, for example one for data and one for /var/www.

In my docker-compose setup I'd do something like:

services:
  littlelink-custom:
    image: julianprieber/littlelink-custom
    volumes:
      - data:/var/data
      - app:/var/www/app
      - database:/var/www/app/data

However I don't know which paths I should pay attention to. Can you enlist what the project uses as folders and files, @JulianPrieber ?

[LeoColman]

In the 'copying the whole application': Nextcloud does that as well, it puts everything in /var/www/html and you can still add volumes to it.

More on that in https://hub.docker.com/_/nextcloud >> Persistent data

docker run -d \
	-v nextcloud:/var/www/html \
	-v apps:/var/www/html/custom_apps \
	-v config:/var/www/html/config \
	-v data:/var/www/html/data \
	-v theme:/var/www/html/themes/<YOUR_CUSTOM_THEME> \
	nextcloud

[JulianPrieber]

However I don't know which paths I should pay attention to. Can you enlist what the project uses as folders and files, [...] ?

@LeoColman Everything is saved in the /htdocs folder including the database.