[FIXED] Fix for canvas installation via Docker on Linux Ubuntu. Also adds Nginx as a proxy to access Canvas via localhost.

[mr-n30]

Summary:

When running the script ./script/docker_dev_setup.sh on Ubuntu 20.04 with 200 GB for storage and 16GB RAM the error below is returned:

> [email protected]" has incorrect peer dependency "[email protected]".
warning " > @storybook/[email protected]" has unmet peer dependency "require-from-string@^2.0.2".
warning " > @instructure/[email protected]" has incorrect peer dependency "webpack@^3 || ^4".
warning "workspace-aggregator-4228077c-1de3-4c6b-b32d-7d1def137de3 > @instructure/canvas-rce > [email protected]" has incorrect peer dependency "axe-core@>=2.2.3 <4".
error Could not write file "/usr/src/app/yarn-error.log": "EACCES: permission denied, open '/usr/src/app/yarn-error.log'"
error An unexpected error occurred: "EACCES: permission denied, mkdir '/usr/src/app/packages/babel-plugin-themeable-styles/node_modules'".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
  /o\ Something went wrong. Check /home/ubuntu/canvas-lms/log/docker_dev_setup.log for details.

This issue is with yarn not being able to write to certain directories and files. During the installation process before yarn install is called the owner and permissions are changed which causes this error. I've added a fix for Ubuntu 20.04 host operating systems in the Additional notes section of this issue.

Steps to reproduce:

1. Install Ubuntu Server 20.04 on a VM with 200GB storage and 16GB RAM
2. Clone canvas-lms repo from GitHub and run: ./script/docker_dev_setup.sh
3. See that you get the error message from above

Expected behavior:

No errors should occur

Actual behavior:

Permission errors via yarn install

Additional notes:

Below are the steps I took to fix the issues:

Replace the lines:

diff --git a/Dockerfile b/Dockerfile
index bda70372d8..a61e8f3ef5 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -29,6 +29,16 @@ WORKDIR $APP_HOME
 
 USER root
 
+# Add docker to group sudo so we can use
+# `sudo chown` later in yarn install and
+# prevent permission issues.
+# Create a new user with sudo privileges
+RUN useradd -ms /bin/bash docker || \
+    usermod -aG sudo docker
+# Set a password for the new user
+# Not a good idea to hard code passwords!!
+RUN echo 'docker:password' | chpasswd
+
 ARG USER_ID
 # This step allows docker to write files to a host-mounted volume with the correct user permissions.
 # Without it, some linux distributions are unable to write at all to the host mounted volume.
diff --git a/script/install_assets.sh b/script/install_assets.sh
index 05630c5332..330018b278 100755
--- a/script/install_assets.sh
+++ b/script/install_assets.sh
@@ -14,11 +14,14 @@ function bundle_config_and_install() {
 
 function yarn_install() {
   echo "Running yarn install..."
+  echo 'password' | sudo -S chown docker:docker /usr/src/app -R # This fixes permission issues. Somewhere /usr/src/app is being changed to something else that docker can't yarn can't write to.
   yarn install --ignore-optional || yarn install --ignore-optional --network-concurrency 1
 }
 
 function compile_assets() {
   echo "Running compile assets dev (css and js only, no docs or styleguide)..."
+  bundle update # This fixes other errors generated after yarn install completes
+  bundle install
   bundle exec rails canvas:compile_assets_dev
 }

Next in your terminal run:

touch Gemfile.rails70.lock.partial

Now you should be able to run ./script/docker_dev_setup.sh without any issues:

./script/docker_dev_setup.sh

After everything finished I didn't select to install dory so I wasn't able to access the docker container from outside the server. What I did to fix that was the steps below:

Install nginx:

sudo apt install nginx

Modify the http section in /etc/nginx/nginx.conf:

server {
    listen 80;
    server_name canvas.docker;
    location / {
        proxy_pass http://<internal_docker_private_ip>:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Then in your terminal run:

sudo nginx -t

and

sudo service nginx reload

Next modify your /etc/hosts (Not the VM but your actual host) to be something like:

<VM_IP>	canvas.docker canvas

You should now be able to access canvas with:

curl http://canvas.docker/

A temporary fix if you get this error below when trying to install canvas via docker_dev_setup.sh:

/usr/src/app/Gemfile.rails70.plugins.lock. It is likely that you need to grant
write permissions for that path.

Modify the files below and then run docker_dev_setup.sh again if you haven't already:

diff --git a/Dockerfile b/Dockerfile
index 6484bf5a1f..050c3211ac 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -29,6 +29,12 @@ WORKDIR $APP_HOME
 
 USER root
 
+# This is required in order to change the permissions and 
+# ownership of the directory that causes permission issues
+# via bundle_config_and_install() in install_assets.sh
+RUN useradd -ms /bin/bash docker || usermod -aG sudo docker
+RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
+
 ARG USER_ID
 # This step allows docker to write files to a host-mounted volume with the correct user permissions.
 # Without it, some linux distributions are unable to write at all to the host mounted volume.
diff --git a/script/install_assets.sh b/script/install_assets.sh
index 05630c5332..e09067e507 100755
--- a/script/install_assets.sh
+++ b/script/install_assets.sh
@@ -9,6 +9,7 @@ function bundle_config_and_install() {
   bundle config --global build.nokogiri --use-system-libraries &&
   bundle config --global build.ffi --enable-system-libffi &&
   mkdir -p /home/docker/.bundle &&
+  sudo chown -R docker:docker /usr/src/app
   bundle install --jobs $(nproc)
 }
 

Then you should be able to access canvas after following the steps here to install nginx Install nginx

This should fix the issue for now. The main issue is somewhere in the scripts/ directory and permissions being changed during install.

[DevGamer9991]

Thanks for showing how to fix it, I had been stuck on this for a week now.

[mr-n30]

A temporary fix if you get this error below when trying to install canvas via docker_dev_setup.sh:

/usr/src/app/Gemfile.rails70.plugins.lock. It is likely that you need to grant
write permissions for that path.

Modify the files below and then run docker_dev_setup.sh again if you haven't already:

diff --git a/Dockerfile b/Dockerfile
index 6484bf5a1f..050c3211ac 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -29,6 +29,12 @@ WORKDIR $APP_HOME
 
 USER root
 
+# This is required in order to change the permissions and 
+# ownership of the directory that causes permission issues
+# via bundle_config_and_install() in install_assets.sh
+RUN useradd -ms /bin/bash docker || usermod -aG sudo docker
+RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
+
 ARG USER_ID
 # This step allows docker to write files to a host-mounted volume with the correct user permissions.
 # Without it, some linux distributions are unable to write at all to the host mounted volume.
diff --git a/script/install_assets.sh b/script/install_assets.sh
index 05630c5332..e09067e507 100755
--- a/script/install_assets.sh
+++ b/script/install_assets.sh
@@ -9,6 +9,7 @@ function bundle_config_and_install() {
   bundle config --global build.nokogiri --use-system-libraries &&
   bundle config --global build.ffi --enable-system-libffi &&
   mkdir -p /home/docker/.bundle &&
+  sudo chown -R docker:docker /usr/src/app
   bundle install --jobs $(nproc)
 }
 

Then you should be able to access canvas after following the steps here to install nginx Install nginx:.

This should fix the issue for now. The main issue is somewhere in the scripts/ directory and permissions being changed during install.

[Kantkun]

@mr-n30 thank you.

[Yetoo1]

After I applied the changes in https://github.com/instructure/canvas-lms/issues/2199#issuecomment-1565682018 (I'm not sure if the changes and/or touch Gemfile.rails70.lock.partial need to be executed, I didn't use either I just used what is in the aforementioned comment) I was able to run the docker and connect to the server, but when I tried pulling and merging changes I found most of the files and dirs in the cloned repo including .git changed user:group with 9999:9999 and I get permission denied. I may be able to get around this by adding my user to the 9999 group, but haven't tried this yet, just wanted to provide feedback.

EDIT: I tried adding user to 9999, but I get group does not exist. I try sudo groupadd 9999, but I get 9999 is not a valid name.

Additional notes:

1. If I change all user:group permissions back to standard user in cloned repo and pull and merge changes I get permission denied in /usr/src/app (I think) when the bundle install step happens with docker_dev_update.sh.

[wallarug]

Can this fix be pushed upstream?

Why has this not been incorporated into a PR and fixed up permanently?

I came here from #2212

[Yetoo1]

@wallarug Probably because of permission issues when updating that I discussed in https://github.com/instructure/canvas-lms/issues/2199#issuecomment-1588558493 above. Do you not experience this?

[wallarug]

@wallarug Probably because of permission issues when updating that I discussed in #2199 (comment) above. Do you not experience this?

The fix worked fine for me (after fixing up my own typos). Would be good to open a PR for this and fix it.

[Yetoo1]

When I was reading the quick start I was just reading https://github.com/instructure/canvas-lms/blob/master/doc/docker/developing_with_docker.md and not https://github.com/instructure/canvas-lms/blob/master/doc/docker/ which lists the commands to change permissions and other setup operations. After I ran the commands and followed the instructions at https://github.com/instructure/canvas-lms/blob/master/doc/docker/ on the canvas clone I didn't have the canvas permission issues pulling/updating.

[Launchable-AI]

A temporary fix if you get this error below when trying to install canvas via docker_dev_setup.sh:

/usr/src/app/Gemfile.rails70.plugins.lock. It is likely that you need to grant
write permissions for that path.

Modify the files below and then run docker_dev_setup.sh again if you haven't already:

diff --git a/Dockerfile b/Dockerfile
index 6484bf5a1f..050c3211ac 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -29,6 +29,12 @@ WORKDIR $APP_HOME
 
 USER root
 
+# This is required in order to change the permissions and 
+# ownership of the directory that causes permission issues
+# via bundle_config_and_install() in install_assets.sh
+RUN useradd -ms /bin/bash docker || usermod -aG sudo docker
+RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
+
 ARG USER_ID
 # This step allows docker to write files to a host-mounted volume with the correct user permissions.
 # Without it, some linux distributions are unable to write at all to the host mounted volume.
diff --git a/script/install_assets.sh b/script/install_assets.sh
index 05630c5332..e09067e507 100755
--- a/script/install_assets.sh
+++ b/script/install_assets.sh
@@ -9,6 +9,7 @@ function bundle_config_and_install() {
   bundle config --global build.nokogiri --use-system-libraries &&
   bundle config --global build.ffi --enable-system-libffi &&
   mkdir -p /home/docker/.bundle &&
+  sudo chown -R docker:docker /usr/src/app
   bundle install --jobs $(nproc)
 }
 

Then you should be able to access canvas after following the steps here to install nginx Install nginx:.

This should fix the issue for now. The main issue is somewhere in the scripts/ directory and permissions being changed during install.

Thanks so much for this! I've been stuck on installing this for 2 days (Ubuntu 22.04 on AWS, in case anyone else is on same)

[Yetoo1]

@mr-n30 I recommend making a pull request with these patches to get a maintainer to do something about it even if there is a security/correctness issue (I'm not sure if there is). If you're not going to make it soon, then I'm going to make it.

[mr-n30]

@Yetoo1 https://github.com/instructure/canvas-lms/pull/2227

[xeruf]

some things were incorporated here, but I still get errors