Docker intergration

[programster]

This pull request aims to simplify the deployment of polr by adding support for docker. I have managed to do this by just adding a docker folder with all the necessary scripts/configuration files. The docker container will keep the state in a volume, so that state will persist across containers coming and going.

I have made a demonstration video of how it works.

The container uses Ubuntu 16.04 rather than something like alpine, so that it is easier for the community to maintain/update should the need arise.

[programster]

Looking at the travis output, it looks like the test against 5.6 passed but 7.0 failed? This is wierd because I haven't changed any of the main code, literally just added a folder called docker that the application doesn't use.

[cydrobolt]

Have you looked at this existing Polr Docker file by another user? https://hub.docker.com/r/ansgar78/polr/ I'm not sure why Travis is failing. Looks like it could be a problem on their end. I'll restart the build.

[cydrobolt]

Have you tested this Dockerfile? Does it work correctly?

[programster]

Firstly on testing. I have ran the build and deployment as demonstrated in that video I linked. I was able to perform the setup, login, as well as create public and private links which worked for me. I was also able to kill and remove the container before re-deploying without losing any state (the links I had put in). If there are further tests you wish me to perform, I would be happy to.

I did see that dockerfile you sent me, but I thought this pull request was necessary, primarily for the fact that I believe docker integration should be within the projects themselves rather than outside it. This way there is an "officially supported" way to deploy the service that can be tested against by the maintainers (all on the same page sort of thing). If a user chooses to run a different setup that doesn't work, that's on them. I don't actually care what the environment is (you can use alpine, ubuntu, php5 or 7 etc), just that there is one that is officially supported/maintained by the developers of the project and it will be updated as necessary with the project. For example, if the project decided to use a feature available only in php 7 (such as specifying return types), then all they have to do is make sure the dockerfile uses php7. Then when someone builds/pulls the latest image, their system will automatically use the correct environment.

The last build of the linked dockerhub image was 9 months ago. Also, one cannot use that dockerfile as it would not work in this project. It has the following line:

COPY "conf/nginx.conf" "/etc/nginx/nginx.conf"

I could not see the conf folder with an nginx.conf file in it. Perhaps I missed it but I suspect that the project had this 9 months ago (reinforcing my previous point for the need for the docker build code to be kept within the project), or the developer of this image is using some added code for their build that others don't have access to if they wished to build their own image with the same dockerfile.

Finally, I wish to reinforce that although my submitted pull request uses Ubuntu 16.04 with php 7, and apache, I would be more than happy if someone was to change this to alpine with php 5 and nginx. All I care about is that it is maintained within the project, and I suspect the contributors to this project will have an easier time updating a Dockerfile based on Ubuntu than alpine simply due to the widespread previous experience/usage of ubuntu in normal non-container based usage, and widespread tutorials on getting things set up in Ubuntu.

[programster]

Is there something about the implementation I need to change, such as using nginx instead of apache, alpine instead of ubuntu, or perhaps the project just does not wish for docker integration?

[cydrobolt]

Neither! We're all rather busy with our daily lives and can't always review pull requests or comments immediately. I'll let you know once I get the time to look through the PR and test it locally.

Thank you for your contribution!

[cydrobolt]

I left a comment on one of the files. Additionally, if we were to merge this into our repository, we would need to provide proper documentation for its use on the docs website, which is within the "docs" folder of this repo. I can wrote the documentation if you can answer the question I left as a comment.

Cachet's documentation on Docker could be useful in crafting our own. https://docs.cachethq.io/docs/get-started-with-docker

[programster]

With regards to documentation with deploying with docker, it should be fairly simple and I would be happy to re-submit the pull request with an updated README, but I get the feeling you would do a better job than I, with updating the docs area.

The hardest part is that because it doesn't (yet) use a MySQL container and a compose file, the user needs to deploy a MySQL database somewhere (usually on the same host), and comment out the bind-address statement so the mysql database doesn't just listen to localhost. They then just need to log into it and create a database with any name. E.g

create database polr

Other than than that, they just need to run the build and deploy scripts in the docker folder:

cd docker
bash build.sh
bash deploy.sh

The deployment script doesn't just call docker run with the appropriate port, it also makes sure to set up a folder structure for the volume which state is to be kept in if it doesn't exist.

Then just feed in the database details, such as the host (which can't be localhost but could be the same server's IP or hostname), database name, user, and password into the web interface when prompted.

[cydrobolt]

I tried building your image on my laptop (Fedora 25 with SELinux enforcing), but I'm encountering many SELinux problems. If we want to use an external directory to store files, we should probably set proper SELinux contexts on the folders.

Here's an example of one of the SELinux denials I'm encountering:

SELinux is preventing chmod from setattr access on the directory polr-data.

*****  Plugin catchall (100. confidence) suggests   **************************

If you believe that chmod should be allowed setattr access on the polr-data directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'chmod' --raw | audit2allow -M my-chmod
# semodule -X 300 -i my-chmod.pp

Additional Information:
Source Context                system_u:system_r:container_t:s0:c251,c489
Target Context                unconfined_u:object_r:user_home_t:s0
Target Objects                polr-data [ dir ]
Source                        chmod
Source Path                   chmod
Port                          <Unknown>
Host                          pyrope
Source RPM Packages           
Target RPM Packages           
Policy RPM                    selinux-policy-3.13.1-225.11.fc25.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     pyrope
Platform                      Linux pyrope 4.8.12-300.fc25.x86_64 #1 SMP Fri Dec
                              2 17:52:11 UTC 2016 x86_64 x86_64
Alert Count                   1
First Seen                    2017-05-27 12:41:17 EDT
Last Seen                     2017-05-27 12:41:17 EDT
Local ID                      6caa895b-e832-42b3-b090-4f79d34e69a4

Raw Audit Messages
type=AVC msg=audit(1495903277.32:305): avc:  denied  { setattr } for  pid=5966 comm="chmod" name="polr-data" dev="dm-2" ino=9189378 scontext=system_u:system_r:container_t:s0:c251,c489 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=0


Hash: chmod,container_t,user_home_t,dir,setattr

[programster]

@cydrobolt I'm afraid that I'm out of my depth here (with SELinux) as I haven't used CentOS in years (ubuntu/debian based which doesn't have it), and back then the solution was always to dsiable selinux which is pretty poor. Would having the volumes where the state are stored being elsewhere on the system resolve the issues selinux is having and be an easier fix? I will look into installing a fedora VM and how to set SELinux contexts.

[James9074]

@programster Thanks for your work on this one! I've put up a dockerhub repo based on your latest docker branch, and wrote a little doc on how to deploy it quickly (none of the other images on DockerHub seem to work out of the box on Ubuntu 16.04).

The docker-compose file isn't configured to build the project yet, though. If I do that I'll submit a PR to your docker branch. I'll also probably want to work with Polr so we can use environment variables or secrets for configuring the db. For now, there's a lightweight docker-compose file on the DockerHub page that will allow you to rapidly deploy both Polr and a MySQL instance linked together.

Hopefully this will make it even easier for people to get some Polr goodness going on. I dropped that docker-compose file in my rancher cluster, and Polr was live and ready to go in about 15 seconds:

The docker-compose.yml:

version: '2'
services:
  polr:
    image: james9074/polr:latest
    volumes:
    - /data/polr/web:/data
    links:
    - db:db
    ports:
    - 80:80/tcp
  db:
    image: mysql
    environment:
      MYSQL_DATABASE: polr
      MYSQL_PASSWORD: some-polr-password
      MYSQL_ROOT_PASSWORD: some-super-secure-pw
      MYSQL_USER: polr
    volumes:
    - /data/polr/db:/var/lib/mysql

[programster]

@James9074 thats awesome. I deliberately didn't touch the database side for now but using another container for the database and using a compose file to link them definitely feels like the right way to go. I'm just trying to tackle the application layer and pushing the image for now. Agree with environment varaibles and secrets. I haven't looked at rancher in over a year I believe. Blown away by the "See Rancher in Action" demo video. That looks like something super useful.

[James9074]

@programster Oh yeah, I've replaced ansible, puppet, chef, even vagrant almost everywhere both at home and at work with Rancher (or dcos/mesos). It's allowing me to orchestrate, manage, and scale like 15 production apps by myself.

This dockerfile has the env stuff set up, but it's not complete and it doesn't seem to work out of the box. It makes sense though as a good starting point though.

[cydrobolt]

@programster perhaps this will help? http://www.projectatomic.io/blog/2015/06/using-volumes-with-docker-can-cause-problems-with-selinux/

It looks like docker run has some parameters that can be used to automatically set the proper context for the volumes.

[ninthwalker]

Just stopping by to say thanks for the docker build you guys have so far. works well.

[overint]

@cydrobolt

What are your thoughts on getting this merged?

[programster]

Hey guys, wow I completely forgot about this merge request I made ages ago. This easter weekend would be a great time for me to take @fagiani comments into account, and update the code from upstream and re-submit the pull request. I'll also take into account the PR quality review comments.

Does that sound like a plan?

[overint]

That sounds awesome! Thanks so much :)

[zubinraj]

+1 for the docker integration.

(fyi) Stumbled on another repository that integrated polr with docker https://github.com/sahil87/polr (I'm not associated with this repository in any way)

[James9074]

@zubinraj https://hub.docker.com/r/james9074/polr/ is a working docker image btw - no github repo but a compose that works out of the box (and in Rancher!). It's kind of old though, if anyone wants me to update it let me know.

[programster]

Hey, So I have switched it over to using docker-compose to spin up a mysql container linked to the polr container. It uses auto-generated passwords that the user never even has to see because they will be auto-populated into the setup form page.

To run polr now, all the user has to do is:

• clone the repository
• navigate to the docker folder
• run bash configure.sh to generate passwords and environment files.
• run bash build.sh to build the polr image
• run bash deploy.sh which will run docker-compose up and deploy the mysql container linked to the polr container (and have all state stored in the polr-data folder).

I made a video demonstrating it working with the changes: https://www.youtube.com/watch?v=gySD3FuRr1U&feature=youtu.be

[overint]

run bash configure.sh to generate passwords and environment files. run bash build.sh to build the polr image run bash deploy.sh which will run docker-compose up and deploy the mysql container linked to the polr container (and have all state stored in the polr-data folder).

Any chance we could have a single file that executes all the commands? Like ./docker/init

That runs all the required command?

[programster]

@overint I agree that having 3 scripts really sucks. I could probably merge the configure and deploy scripts, and just have the configure script part check if it has already run and just not run if it has (changing the passwords loses you access to your database).

However, you would not want to merge the deploy.sh and build.sh scripts as you only want to run a build whenever you perform a code update, or just to have an updated container every couple weeks. The deploy command you would want to run whenever your containers die for whatever reason or after a reboot etc. I guess you could have deploy.sh and buildAndDeploy.sh scripts though, as you probably want to deploy after every build. Maybe in future there will be an official polr image, and people will never need to run the build, the deploy script will just pull it from dockerhub.

Actually I'll have just the one init script and have it just have it ask the user questions such as do they want to build the container (or just building it if they dont have a local image). Once we are happy with what it does, we can have it take arguments to automatically answer those questions. E.g. --no-build etc.

[programster]

I'm hoping all of those changes address everything that was raised. If I've missed something, please let me know. If you are all happy, then we can put the instructions into the README or perhaps stick a docker-specific README in that folder?

[navidurrahman]

My Two cents:

I was trying to use polr docker container in the multi-node environment, My Question is; What is the point of env file when you are mounting the .env file from the volume itself? Also, Loading configs from the environment are pretty standard in docker environment. I was expecting the same behaviour from this container.

Running this docker container on a large scale docker orchestration tool like Kubernetes, Mesos or ECS is tedious, as it will try to create a polr-data folder on each machine to maintain its state, which is essentially not an ideal solution.

The whole idea of mounting file system into the container make the app stateful, which can be resolved by creating the env file from the environment.

Example: https://github.com/CachetHQ/Docker/blob/master/entrypoint.sh

[zubinraj]

@James9074 - Thank you for the reference. I have pulled the latest from @cydrobolt /master and have created a docker compose based image here - https://hub.docker.com/r/zubinraj/polr/

[pascalandy]

I'll be glad to help testing the docker image :)

[tkaefer]

Sorry for hijacking this thread: I've setup a Alpine+nginx+fpm based Polr Docker deployment. If anybody is interested, just check https://github.com/tkaefer/polr

[dan-kez]

Hi all! Just wanted to check in and see if there are any blocking concerns before merging this in. It would definitely expedite setup for new users

[ralyodio]

Is this issue going to happen soon?