Failed to open stream: Permission denied

[jaydrogers]

Affected Docker Images

All

Current Behavior

There are many reports where this error randomly appears, usually on fresh deployments.

Failed to open stream: Permission denied

Examples

https://github.com/serversideup/docker-php/pull/172#issue-1654792968

Bugflow

Expected Behavior

There is a discussion where we may be able to adjust S6 Overlay and have the commands run correctly (instead of root)

https://github.com/serversideup/docker-php/discussions/71#discussioncomment-5841649

Steps To Reproduce

TBD

Host Operating System

Linux

Docker Version

N/A

Anything else?

No response

[jaydrogers]

I am re-opening this issue.

This is related to the attempted fix implemented in #180, suggested by @tsterker.

Reason for re-open

• I am experiencing permission issues in GitLab CI causing builds to fail

More details on the failure

1. GitLab CI + DIND (Docker-in-docker) uses root and the UID of 0. With the commands using UID 9999 by default, the CI machine cannot create a vendor directory on the first step of composer install

Ways to test your own apps

• Use the beta images on your app and see if you get the same results (https://serversideup.net/open-source/docker-php/docs/getting-started/contributing#beta-images)

Next steps

I am going to continue to troubleshoot and may possibly revert #180 if I cannot get this to work well.

[jaydrogers]

Adding my error message in here too for future reference

[danilopolani]

Wanted to add a few things on this issue.
I copied all the project files with the proper user ownership like this:

COPY --chown=$PUID:$PGID ./composer.json ./composer.lock ./
RUN composer install --no-cache --no-dev --no-scripts --no-autoloader --ansi --no-interaction

COPY --chown=$PUID:$PGID . .
RUN composer dump-autoload -o

Now the issue is that it can't create cache files:

Unable to create lockable file: /var/www/html/storage/framework/cache/data/
fa/f2/faf26108316c685830630fa2070ff0a7b212a06a. Please ensure you have perm
ission to create files in this location.

I've tried giving to the whole storage folder all the permissions and ownership to both $PUID:$PGID or www-data:www-data, but no luck so far. Also with the beta I got no luck. What's the user running the webserver?

[jaydrogers]

Thanks for chiming in. The user running PHP is webuser, so you might need to do a su before running your composer commands.

Also -- I am not sure why you would need two copy commands in there. Also, the default web root is /var/www/html. If you're running your build within the Docker build process, then it would look something like this:

COPY --chown=$PUID:$PGID . /var/www/html
RUN su webuser && \
    composer install --no-cache --no-dev --no-scripts --no-autoloader --ansi --no-interaction && \
    composer dump-autoload -o

[danilopolani]

Right, splitting the command is not needed. At the end I've cleared all my docker images and containers and now this is the full, working code, no errors related to laravel.log nor cache creation, if it may help someone else too.

FROM serversideup/php:8.2-fpm-nginx as base

ENV AUTORUN_LARAVEL_MIGRATION=true
ENV SSL_MODE=off
ENV APP_ENV=production

FROM base as production

COPY --chown=$PUID:$PGID . .

RUN composer install --no-cache --no-dev --no-scripts --no-autoloader --ansi --no-interaction \
    && composer dump-autoload -o

I had to create a separate Dockerfile because I added (not pasted here) some tweaks to opcache, more extensions and so on. Thank you for the support and the image :)

[tsterker]

@jaydrogers I'm currently trying to follow up on this topic and understand the issue mentioned above and how #180 could be the cause for this.

ℹ️ Is there an example that works with the current setup and breaks with #180?

e.g. you mention a GitLab CI + DIND setup above.

GitLab CI + DIND (Docker-in-docker) uses root and the UID of 0. With the commands using UID 9999 by default, the CI machine cannot create a vendor directory on the first step of composer install

Is this related to building docker images? In this case, maybe kaniko could be an option to build images without privileged mode?

That said, if there is a general compatibility issue with the ubiquitous DIND it's still a problem that would need solving.

[jaydrogers]

Hey @tsterker,

Thanks for checking in!

Regarding where the issue stands now

The root cause of his issue is a user experience issue -- it's not the users fault either.

Some background:

• PHP-FPM needs root to start its master process, but that's unsafe to run a PHP app as root all the time
• We add webuser to bring those privileges down to a non-privileged user

Where the problem stems:

• When a user runs "composer install" (for example, docker run -v $(pwd):/var/ww/html serversideup/php:8.2-fpm-nginx composer install) this will execute the command as root

The user would need to know to switch to webuser first, then execute the command (which is a poor user experience). The above command usually creates a "domino effect" error where the Laravel log gets created as root then the webserver runs into a second error, but the web user is trying to write to the log as webuser (thus throwing the permission denied error).

Regarding the CI + DIND

Is this related to building docker images? In this case, maybe kaniko could be an option to build images without privileged mode?

I want the images to assume this is compatible with any system and I don't want any dependencies on other libraries.

Moving forward

Let me take some time to document the issue better and see if I can post an issue on the S6 Overlay repo. They've been really helpful in the past, but I want to make sure I have the issue fully documented so I don't waste any of their time.

I'll keep you posted!

[jaydrogers]

Just adding a note for myself...

S6 Overlay has some changes that were released on May 4th. I might want to explore these changes since they talk specifically about the USER directive: https://github.com/just-containers/s6-overlay#notes

[jaydrogers]

Just adding a note, https://github.com/serversideup/docker-php/pull/180 has been reverted.

New approach

• I am going to explore if the USER directive in v3.1.5.0 of S6 Overlay is a better approach

[elcapo]

I'm having the same issue here. Just for context, if I check the owner of the files using a disposable container by running spin run php ls -la, I get a list of files owned by 1000:1000:

127.0.0.1 - - [10/Aug/2023:15:45:19 +0000] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"
total 412
drwxrwxr-x 15 1000 1000   4096 Aug 10 15:26 .
drwxr-xr-x  1 root root   4096 Aug  8 08:16 ..
-rw-rw-r--  1 1000 1000    258 Jun 30 15:18 .editorconfig
-rw-rw-r--  1 1000 1000   1197 Aug 10 15:07 .env
-rw-rw-r--  1 1000 1000   1099 Aug 10 15:07 .env.example
drwxrwxr-x  7 1000 1000   4096 Aug 10 14:52 .git
-rw-rw-r--  1 1000 1000    186 Jun 30 15:18 .gitattributes
-rw-rw-r--  1 1000 1000    255 Aug 10 14:38 .gitignore
-rw-rw-r--  1 1000 1000   4158 Jun 30 15:18 README.md
drwxrwxr-x  3 1000  998   4096 Aug 10 15:12 _volumes
drwxrwxr-x  7 1000 1000   4096 Jun 30 15:18 app
-rwxr-xr-x  1 1000 1000   1686 Jun 30 15:18 artisan
drwxrwxr-x  3 1000 1000   4096 Jun 30 15:18 bootstrap
-rw-rw-r--  1 1000 1000   1919 Aug 10 14:32 composer.json
-rw-rw-r--  1 1000 1000 294977 Aug 10 14:32 composer.lock
drwxrwxr-x  2 1000 1000   4096 Jun 30 15:18 config
drwxrwxr-x  5 1000 1000   4096 Jun 30 15:18 database
drwxrwxr-x  2 1000  998   4096 Aug 10 15:27 docker
-rw-rw-r--  1 1000 1000   1080 Aug 10 15:14 docker-compose.dev.yml
-rw-rw-r--  1 1000 1000    279 Aug 10 15:30 docker-compose.yml
-rw-rw-r--  1 1000 1000    248 Jun 30 15:18 package.json
-rw-rw-r--  1 1000 1000   1084 Jun 30 15:18 phpunit.xml
drwxrwxr-x  2 1000 1000   4096 Jun 30 15:18 public
drwxrwxr-x  5 1000 1000   4096 Jun 30 15:18 resources
drwxrwxr-x  2 1000 1000   4096 Jun 30 15:18 routes
drwxrwxr-x  5 1000 1000   4096 Jun 30 15:18 storage
drwxrwxr-x  4 1000 1000   4096 Jun 30 15:18 tests
drwxrwxr-x 40 1000 1000   4096 Aug 10 14:32 vendor
-rw-rw-r--  1 1000 1000    263 Jun 30 15:18 vite.config.js

Which is interesting because I expected the user and group ids to be 9999:9999.

[elcapo]

Which is interesting because I expected the user and group ids to be 9999:9999.

For additional context, I tried to run the project changing the 9999 user and group ids to 1000 (the default values) and the permission denied error disappeared. Briefly, I replaced 9999 with 1000 in:

• https://github.com/serversideup/docker-php/blob/290e14f973572ef1ad0a25acb43a88d1dfc19d1e/src/cli/Dockerfile#L30
• https://github.com/serversideup/docker-php/blob/290e14f973572ef1ad0a25acb43a88d1dfc19d1e/src/cli/etc/s6-overlay/scripts/runas-user#L4
• https://github.com/serversideup/docker-php/blob/290e14f973572ef1ad0a25acb43a88d1dfc19d1e/docs/content/docs/6.reference/1.environment-variable-specification.md?plain=1#L15

Actually, if I now run a ls -la command of the webroot, now I get:

127.0.0.1 - - [10/Aug/2023:16:22:56 +0000] "GET /ping HTTP/1.1" 301 162 "-" "curl/7.81.0"
total 412
drwxrwxr-x 15 webuser webgroup   4096 Aug 10 15:26 .
drwxr-xr-x  1 root    root       4096 Aug 10 16:10 ..
-rw-rw-r--  1 webuser webgroup    258 Jun 30 15:18 .editorconfig
-rw-rw-r--  1 webuser webgroup   1197 Aug 10 15:07 .env
-rw-rw-r--  1 webuser webgroup   1099 Aug 10 15:07 .env.example
drwxrwxr-x  7 webuser webgroup   4096 Aug 10 14:52 .git
-rw-rw-r--  1 webuser webgroup    186 Jun 30 15:18 .gitattributes
-rw-rw-r--  1 webuser webgroup    255 Aug 10 14:38 .gitignore
-rw-rw-r--  1 webuser webgroup   4158 Jun 30 15:18 README.md
drwxrwxr-x  3 webuser      998   4096 Aug 10 15:12 _volumes
drwxrwxr-x  7 webuser webgroup   4096 Jun 30 15:18 app
-rwxr-xr-x  1 webuser webgroup   1686 Jun 30 15:18 artisan
drwxrwxr-x  3 webuser webgroup   4096 Jun 30 15:18 bootstrap
-rw-rw-r--  1 webuser webgroup   1919 Aug 10 14:32 composer.json
-rw-rw-r--  1 webuser webgroup 294977 Aug 10 14:32 composer.lock
drwxrwxr-x  2 webuser webgroup   4096 Jun 30 15:18 config
drwxrwxr-x  5 webuser webgroup   4096 Jun 30 15:18 database
drwxrwxr-x  3 webuser      998   4096 Aug 10 15:53 docker
-rw-rw-r--  1 webuser webgroup   1080 Aug 10 15:14 docker-compose.dev.yml
-rw-rw-r--  1 webuser webgroup    212 Aug 10 16:17 docker-compose.yml
-rw-rw-r--  1 webuser webgroup    248 Jun 30 15:18 package.json
-rw-rw-r--  1 webuser webgroup   1084 Jun 30 15:18 phpunit.xml
drwxrwxr-x  2 webuser webgroup   4096 Aug 10 16:17 public
drwxrwxr-x  5 webuser webgroup   4096 Jun 30 15:18 resources
drwxrwxr-x  2 webuser webgroup   4096 Jun 30 15:18 routes
drwxrwxr-x  5 webuser webgroup   4096 Jun 30 15:18 storage
drwxrwxr-x  4 webuser webgroup   4096 Jun 30 15:18 tests
drwxrwxr-x 40 webuser webgroup   4096 Aug 10 14:32 vendor
-rw-rw-r--  1 webuser webgroup    263 Jun 30 15:18 vite.config.js

[sneycampos]

For additional context, I tried to run the project changing the 9999 user and group ids to 1000 (the default values) and the permission denied error disappeared. Briefly, I replaced 9999 with 1000 in:

Same effect here, in my case is 1001

[sneycampos]

I am not expert on how the user 9999 is being created but i think we are on the way :)

[ChrisToxz]

Till there is a solid fix, I would just suggest to add the following environment variables, to let the image run on a different PUID & PGID.

docker-compose example:

environment:
      PUID: 1000
      PGID: 1000

[jaydrogers]

👉 Just a note

If you're running this locally, the owner and group IDs will differ based on the host machine.

If you're using a Linux host to run the docker container

• You'll probably get WYSIWYG (meaning a container running as the default 9999 will create the file as 9999 on the host)

If you're using a Windows or macOS host to run the container -- it gets complicated

• Since Docker is not native to these operating systems, Docker Desktop creates a network file share to mount the files to the container, changing the UID & GIDs

Just another level of complexity I am trying to work through 🙃

[zogot]

For me, when I had an issue trying to create a Docker Image that wasn't using volumes my RUN commands were running as root.

I solved it by setting the user to the expected values before the RUN and then importantly, setting back to root as the last command in the Dockerfile.

This is my Dockerfile

ARG PHP_VERSION='8.2'

# ================
# Base Stage
# ================
FROM serversideup/php:${PHP_VERSION}-fpm-nginx as base
ENV AUTORUN_ENABLED=false
ENV SSL_MODE=off

# ================
# Production Stage
# ================
FROM base as production

ENV APP_ENV=production
ENV APP_DEBUG=false

# Required Modules
RUN apt-get update && \
    apt-get install -y php${PHP_VERSION}-pgsql && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* /usr/share/doc/*

USER $PUID:$PGID

# Copy contents.
# - To ignore files or folders, use .dockerignore
COPY --chown=$PUID:$PGID . .

RUN composer install --optimize-autoloader --no-dev --no-interaction --no-progress --ansi

# artisan commands
RUN php ./artisan event:cache && \
    php ./artisan route:cache && \
    php ./artisan view:cache

USER root:root

Note the setting of USER to $PUID:$PGID and then back to root:root for fpm to work correctly.

[jaydrogers]

Quick Question

Can everyone on this thread post their host operating system? I assume it's all Linux?

I am working on resolving this issue. I finally have a Linux development machine to do some testing and reproduction with this issue to finally get it resolved.

[sneycampos]

Quick Question

Can everyone on this thread post their host operating system? I assume it's all Linux?

I am working on resolving this issue. I finally have a Linux development machine to do some testing and reproduction with this issue to finally get it resolved.

Ubuntu 22.04.3 LTS

[ChrisToxz]

Linux, but got team members using WSl & Mac. Workarounds on Linux works fine on WSl, but Mac seems to be more annoying.. At least development wise with files mounted rather than files inside the image.

[jaydrogers]

Hmm... we use macOS exclusively and never run into a problem. Usually with the file mount on macOS, it actually helps prevent the issues from happening.

Do you have steps to reproduce specifically for your Mac? I can run it on my machine and see if I can replicate.

[gertvdv]

Experiencing the issue on Ubuntu 22.04 LTS, fresh install.

For what it's worth; I'm not seeing this problem with Laravel Sail, maybe we can find some inspiration there?

[elcapo]

Can everyone on this thread post their host operating system? I assume it's all Linux?

Another Ubuntu 22.04 LTS here.

[flemming-pr]

Ubuntu 22.04. This error will occur in mounted path like

volumes:
  - ./storage:/var/www/html/storage

as well in docker volumes

volumes:
  - storage:/var/www/html/storage

[jaydrogers]

I have a gut feeling on the problem and the solution for Linux hosts is a little trickier compared to macOS & Windows.

I have some theories and I will revisit this soon. I will keep everyone posted 👍

[jaydrogers]

I think I have an update on how to fix this. Unfortunately this will require a lot of refactoring.

Keep an eye on this PR: https://github.com/serversideup/docker-php/pull/205

Any feedback, please let me know 👍

More to come soon 🚀

[xaimes]

Hi, I have a problem, probably related to this error. The problem occurs on Linux (fedora) and on Windows (WSL2).

1. when I use my local machine to create a Laravel project (composer create-project laravel/laravel example-app) and run docker compose up the following error pops up:

2. when I use the container to create the Laravel project (docker run --rm -v $(pwd):/var/www/html serversideup/php:beta-8.3-fpm-nginx-alpine composer create-project laravel/laravel example-app) then running docker compose up afterwards displays the laravel welcome page, but I cannot edit any files on my local machine.

The problem is probably related to permissions, but because of this, I am not able to use these containers for development.

In older containers, this solution helped: https://github.com/serversideup/docker-php/pull/172#issuecomment-1512779043

But now these variables are no longer available.

Is there a solution to this problem?

This is my docker compose, it's very simple:

version: '3'
services:
    php:
        image: serversideup/php:beta-8.3-fpm-nginx-alpine
        ports:
            - 80:80
        volumes:
            - .:/var/www/html

[sneycampos]

Hi, I have a problem, probably related to this error. The problem occurs on Linux (fedora) and on Windows (WSL2).

Probably these environment variables will be added soon, just noticed a note in the v3 docs:

[xaimes]

Yes, I hope they will be added.

But I wonder how others are using these containers for develpoment? It doesn't work on Windows or Linux. Does it work seamlessly on MacOs? Does everyone only use these containers production-wise? 😃

[tonysm]

@XaiMeS I'm using it on Linux for local dev, but I had to set the PUID and PGID both to 1000 (my host machine's user ID) for it to work. For instance, this is my local docker-compose.yml:

services:
    laravel.test:
        image: serversideup/php:8.2-fpm-nginx
        extra_hosts:
            - 'host.docker.internal:host-gateway'
        ports:
            - '${APP_PORT:-80}:80'
        environment:
            LARAVEL_SAIL: 1
            SSL_MODE: 'off'
            PUID: '${UID:-1000}'
            PGID: '${GID:-1000}'
        volumes:
            - '.:/var/www/html'
        networks:
            - sail
        depends_on:
            - mysql
// ...

[sneycampos]

@XaiMeS I'm using it on Linux for local dev, but I had to set the PUID and PGID both to 1000 (my host machine's user ID) for it to work. For instance, this is my local docker-compose.yml:

These env vars works in v2, not in v3 beta