docker-compose up -d leaves error message.

[gkoerk]

When using docker and specifically running docker-compose up -d, the webserver container is created but throws the following error: Successfully built 016c459fb247 Successfully tagged paperlessmaster_consumer:latest Creating paperlessmaster_webserver_1 ... done Creating paperlessmaster_webserver_1 ... ERROR: for consumer Container "41e1f6d473c5" is unhealthy. ERROR: Encountered errors while bringing up the project.

I've tried running this thing many times with no success. When it does run, the log shows the following errors when attempting to perform chown: chown errors.txt

My docker-compose.yml: docker-compose.yml.txt

My docker-compose.env: docker-compose.env.txt

[danielquinn]

The line ERROR: for consumer Container "41e1f6d473c5" is unhealthy isn't a Paperless message, but a Docker one. It sounds like there might be something wrong specifically with your Docker installation itself. When I googled that message, the only hint I found was reference to someone having two versions of Docker installed at the same time, so maybe that's what's happening here? I'm afraid I don't know enough about the various ways that Docker might break to be useful here.

[gkoerk]

What about the failure to chown the files?

Jerry

On Fri, Jun 29, 2018 at 4:23 AM, Daniel Quinn [email protected] wrote:

The line ERROR: for consumer Container "41e1f6d473c5" is unhealthy isn't a Paperless message, but a Docker one. It sounds like there might be something wrong specifically with your Docker installation itself. When I googled that message, the only hint I found was reference to someone having two versions of Docker installed at the same time, so maybe that's what's happening here? I'm afraid I don't know enough about the various ways that Docker might break to be useful here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/danielquinn/paperless/issues/372#issuecomment-401300360, or mute the thread https://github.com/notifications/unsubscribe-auth/AFAL9_9Nvuoj4m0fjUchi1BOX5EqVJLlks5uBfIpgaJpZM4U8cAO .

-- Thanks,

Jerry

[danielquinn]

I can't think of any reason why chowning a file would fail other than that the machinery between the application, guest OS, and host OS is somehow misaligned. That chown action is performed by scripts/docker-entrypoint.sh and it's supposed to be running as root in the guest OS. Maybe there's a user privilege issue, like the default Docker user isn't root? I'm sorry, I'm afraid I don't know.

Just to confirm though, you added the .txt to the files you posted so they could more easily be read, right? The file names should be docker-compose.yml and docker-compose.env on your machine. I'm not sure what'll happen if they're not named correctly.

If that's all ok, I'd suggest trying to run other Docker applications (preferably with docker-compose) and see things work as expected.

[gkoerk]

That's what's so frustrating. I had paperless working great with my Fujitsu ScanSnap ix500 scanning directly to the consume folder. I also have about a dozen other containers working fine, mostly via docker-compose up -d.But I am using docker on a QNAP NAS. I may try removing it and reinstalling and trying again.

Jerry

[gkoerk]

I removed all containers, images, and volumnes, uninstalled docker and docker-compose and reinstalled them without error. I removed the healthcheck which was failing, causing the consumer to never be created. However, the containers immediately stop due to the failure to chown many files within the container / in the named volumes. Could it be a permissions issue?

[danielquinn]

It definitely sounds like a permissions issue, but I can't imagine how. I mean, every Docker container should start as the root user, right?

[gkoerk]

Yes - should I leave the USERMAP_UID and USERMAP_GID commented out? It failed with the chown errors when commented out. If I try to make them the same as an existing user, it gives an error that the user already exists. When I use brand new values (like 998:998) it still creates the images but the error this time is:

Mapping UID and GID for paperless:paperless to 998:998, usermod: Failed to change ownership of the home directory

[danielquinn]

Ah ok, I think we're getting somewhere now. That user-mapping section of the entrypoint was fiddled with a bunch of different people and there was some debate around this before. I'm going to go ahead and page @addadi and @pitkley as they were instrumental and figuring it out last time, so they may have some insight here.

[gkoerk]

Awesome!!

[pitkley]

Hey @gkoerk. Could you provide the full Docker logs of the webserver-container? That would help in potentially identifying what is going wrong here.

Could you also provide some additional information on what your host OS is, and since you have seem to have built the container yourself the git commit you are on?

Thanks!

[gkoerk]

Docker logs coming up. I didn't use git - I used wget https://github.com/danielquinn/paperless/archive/master.zip and went from there. My host OS is a proprietary Linux variant for QNAP NAS devices. But it comes with "Container Station" which includes installation of both docker & docker-compose. (But do note that the "root" user is named admin and has a UID=0 and GID=0.

Docker Engine: Version = 17.07.0-ce API version = 1.31 Go version = go1.8.3 OS type = linux OS = QTS 4.3.4 (20180528) Architecture = amd64 Kernel version = 4.2.8

Docker-compose: docker-compose version - 1.18.0

Docker Engine Status: Total CPU = 16 Total memory = 67.5 GB Docker root directory = /share/CACHEDEV6_DATA/Container/container-station-data/lib/docker Storage driver = overlay Logging driver = json-file Cgroup driver = cgroupfs

[gkoerk]

docker-compose up -d output.txt docker-log for webserver.txt

[gkoerk]

BTW - That was done without any modification to the .env or .yml files.

[pitkley]

Thanks for the detailed information. My guess right now given the outputs and especially the error message being "Operation not permitted", is that the filesystem Docker stores the container volumes on does simply not support setting permissions or modifying file owners.

Given the "Docker root directory" you have got as output from docker info, can you identify what filesystem that is? According to the Docker docs, the overlay storage driver is only supported on ext4 and xfs.

[gkoerk]

File system is EXT4.

[gkoerk]

Maybe permissions on the filesystem on which Docker stores the container volumes lacks permissions? I haven't had this issue on any of my other dozen or so containers / stacks.

[gkoerk]

Here is the output to show permissions on the Docker root directory.

[pitkley]

Yeah, I'm a bit stumped on this right now. It seems like it could be a lack of permissions, but it would still be surprising given the container is started as UID 0.

You aren't by chance running the GitLab container by sameersbn? The file permission logic was taken from their container initially.

[gkoerk]

I'm not, sorry. Want me to try it?

[gkoerk]

Funny thing is - I had paperless working great in docker sometime back before I needed to backup & remove "Container Station" and reinstall it (which installs docker and docker-compose as well as lxc). I can try each previous version to see if a change caused this.

[pitkley]

Hm, no, that would probably overkill for the time being. For now, can you start the built image as a separate container and try to execute the other commands below? Maybe that gives us some more insight.

qnap:~# docker run --rm --it --entrypoint /bin/sh paperlessmaster_consumer:latest -c /bin/sh
container:/# cd /usr/src/paperless
container:/usr/src/paperless# ls -lan
container:/usr/src/paperless# chown 1000 requirements.txt
container:/usr/src/paperless# chown :1000 requirements.txt

[gkoerk]

I created a new container with a different entrypoint and ran all 4 of your other statements: [/share/Apps/paperless/paperless-master] # docker run -it --entrypoint /bin/sh 78af583de43b -c /bin/sh paperless_webserver_1 /usr/src/paperless/src # cd .. /usr/src/paperless # ls -lan total 24 drwxr-sr-x 5 1000 1000 4096 Jun 29 18:28 . drwxr-xr-x 3 0 0 4096 Jun 29 18:28 .. drwxr-xr-x 2 1000 1000 4096 Jun 29 18:35 data drwxr-xr-x 3 1000 1000 4096 Jun 29 18:35 media -rw-r--r-- 1 1000 1000 944 Jun 19 10:46 requirements.txt drwxr-xr-x 6 1000 1000 4096 Jun 29 18:28 src /usr/src/paperless # chown 1000 requirements.txt chown: requirements.txt: Operation not permitted /usr/src/paperless # chown :1000 requirements.txt chown: requirements.txt: Operation not permitted

[gkoerk]

I thought about setting the PID=0 and GID=0 but that didn't work because the app complains that the user already exists.

[gkoerk]

Maybe you can explain how the container creates the paperless user: why is it necessary, why do the chown at all? Can't it just be run as root or whichever user is passed in? docker run … -u dockeruser

[gkoerk]

Doesn't docker run the image as root (uid 0) anyway? Why the need to create a new user and group? My guess is that building the container in the Dockerfile the commands are operating as root. Shouldn't the chown be executed by the root user within the container?

[pitkley]

The reason for not using root is security: if the Paperless application had e.g. a remote code execution exploit, you'd instantly have root permissions in the container making it easier to cause harm, potentially even to the host running the container.

You are right that the image is run as root, and the chowns are executed as root, it is only the application that is started as paperless, which is why permissions shouldn't be an issue...

A further reason for having a separate user and enabling remapping of the UID and GID of the user is that on the consumer the user will usually mount the consumption and export directories, which they usually want to keep the rights of their personal user for. So if the e.g. have a user with UID/GID 1002 on the host, the container would allow remapping to that UID/GID as to not cause a permission headache when sharing files between the host and the container.

With all that being said, you can of course try to simply run the image as root by disabling the chown and removing the sudos:

diff --git a/scripts/docker-entrypoint.sh b/scripts/docker-entrypoint.sh
index df8df8f..0e65283 100644
--- a/scripts/docker-entrypoint.sh
+++ b/scripts/docker-entrypoint.sh
@@ -39,7 +39,7 @@ set_permissions() {
       } >&2
     done
     # Set permissions for application directory
-    chown -Rh paperless:paperless /usr/src/paperless
+    #chown -Rh paperless:paperless /usr/src/paperless
 }
 
 migrations() {
@@ -50,7 +50,7 @@ migrations() {
     if (set -o noclobber; echo "$$" > "${LOCKFILE}") 2> /dev/null
     then
         trap 'rm -f "${LOCKFILE}"; exit $?' INT TERM EXIT
-        sudo -HEu paperless "/usr/src/paperless/src/manage.py" "migrate"
+        /usr/src/paperless/src/manage.py "migrate"
         rm ${LOCKFILE}
     fi
 }
@@ -99,7 +99,7 @@ if [[ "$1" != "/"* ]]; then
         install_languages "$PAPERLESS_OCR_LANGUAGES"
     fi
 
-    exec sudo -HEu paperless "/usr/src/paperless/src/manage.py" "$@"
+    exec /usr/src/paperless/src/manage.py "$@"
 fi
 
 exec "$@"

[gkoerk]

There's also a chown in the Dockerfile:

chown -Rh paperless:paperless /usr/src/paperless && \

Should that be commented out as well?

[pitkley]

It might be necessary to have that chown to root:root instead, mainly for the executable permissions.

[stgarf]

@gkoerk Basically, perms issue. What else have you tried since the last comment some 8 months ago? If this is working now, would you mind closing the issue? Otherwise, read on:

First, you should config docker to run containers as some other user, e.g. dockerrunner or somesuch. Then, you should make sure the UID/GID of said other user is mapped to running containers such that when you docker run --rm -it <container name> bash and enter the container you can type id paperless and it should return the UID/GID mapping of the user that runs docker (e.g. in my case 1001).

user@big-cloud-server:~$ docker exec -it paperless-web bash
bash-4.4# id paperless
uid=1001(paperless) gid=1001(paperless) groups=1001(paperless),1001(paperless)
bash-4.4# exit
exit

Further problems can also extend from permissions issues on the volumes that you're mounting into the docker container. Make sure those directories are owned locally and accessible by the user running docker, e.g. in my case dockerrunner.

Futher reading on Docker and user may be of use to you... https://medium.com/redbubble/running-a-docker-container-as-a-non-root-user-7d2e00f8ee15

[michalharakal]

I was able to get it running on synology with following patch

index 30eb3a5..9d8fcf1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -6,6 +6,7 @@ LABEL maintainer="The Paperless Project https://github.com/the-paperless-project

 # Copy Pipfiles file, init script and gunicorn.conf
 COPY Pipfile* /usr/src/paperless/
+RUN addgroup -S paperless && adduser -S paperless -G paperless
 COPY scripts/docker-entrypoint.sh /sbin/docker-entrypoint.sh
 COPY scripts/gunicorn.conf /usr/src/paperless/

@@ -48,8 +49,6 @@ RUN apk add --no-cache \
 # Create the consumption directory
     mkdir -p $PAPERLESS_CONSUMPTION_DIR && \
 # Create user
-    addgroup -g 1000 paperless && \
-    adduser -D -u 1000 -G paperless -h /usr/src/paperless paperless && \
     chown -Rh paperless:paperless /usr/src/paperless && \
     mkdir -p $PAPERLESS_EXPORT_DIR && \
 # Setup entrypoint
@@ -65,6 +64,6 @@ CMD ["--help"]
 COPY src/ /usr/src/paperless/src/
 COPY data/ /usr/src/paperless/data/
 COPY media/ /usr/src/paperless/media/
-
+RUN chown -R paperless:paperless /usr/src/paperless/
 # Collect static files
-RUN sudo -HEu paperless /usr/src/paperless/src/manage.py collectstatic --clear --no-input
\ No newline at end of file
+RUN sudo -HEu paperless /usr/src/paperless/src/manage.py collectstatic --clear --no-input
diff --git a/scripts/docker-entrypoint.sh b/scripts/docker-entrypoint.sh
index d245172..2194db7 100644
--- a/scripts/docker-entrypoint.sh
+++ b/scripts/docker-entrypoint.sh
@@ -40,6 +40,8 @@ set_permissions() {
     done
     # Set permissions for application directory
     chown -Rh paperless:paperless /usr/src/paperless
+    chown -R paperless:paperless /consume
+    chmod -R 777 /consume
 }

 migrations() {