Docker support

[bploetz]

This attempts to add support for running Dremio in Docker.

• Added a "run" option to the dremio script that runs the process in the foreground (required by Docker). I'm not married to the name, if you want to change it to "start-fg" and "start-bg" or something to make it more obvious as to what each one does. I would have also updated the docs, but I can't seem to find them...

• Added a Dockerfile for the OSS version. This is based off of a Dockerfile we use for running Dremio in Docker. While I would have liked to base this off of Alpine, I ran into problems with rocksdb libraries in Alpine that I couldn't figure out how to work around. If anyone has an example that works with Alpine I'd love to see it. Also note that this uses the project.version when downloading the distribution, but there's a problem where the value within Maven uses underscores whereas it looks like the artifacts on the cdn look like they use dashes. Not sure how to fix that. But this means the released artifacts must be in place before building the Docker image.

Also, not sure if I've even put the docker module in the right place in the project, so just let me know if that needs to move.

Thanks. BP

[laurentgo]

Thanks for the contribution, we are looking into it. Some quick comments:

• in the next minor version (1.5?), Dremio will include a start-fg command
• Instead of fetching the community version from the download website, it might be better to create the image as part of the build (there are docker plugins for maven).
• 1.4.9 final version number is using underscores for maven because our artifact repository had some trouble with an all numeric git sha :( sorry for the extra pain
• Alpine distro is probably using musl C library, but RocksDB targets glibc, which is why you might experience crashes

[bploetz]

@laurentgo ah ok cool, that's good to know about start-fg.

I'll take a crack at changing this to create the image as part of the build based on the artifacts produced by the build, rather than downloading them. I've only ever used the dockerfile-maven-plugin from spotify, but if you have another Docker maven plugin you prefer let me know.

Thanks.

[laurentgo]

I know about 2 docker plugin, the spotify and the fabric8.io ones, but haven't decided which one to use yet :)

[laurentgo]

The dockerfile seems to assume that the logs would be written under /var/log/dremio, but unless dremio-env is modified, default log location would be /opt/dremio/log (same for data dir and pid dir).

[bploetz]

@laurentgo yeah since the Dockerfile is essentially installing from tar, that whole section was cribbed from the tar ball install instructions (https://docs.dremio.com/deployment/standalone-tarball.html), which honestly didn't make a lot of sense there either. I can certainly remove those and stick with the default /opt/dremio structure.

[bploetz]

@lopezunwired ah, looks like the default user in the ubuntu:16.04 image that we inherit from is root

> docker run -it ubuntu:16.04 /bin/bash
root@df5adfc527a7:/# whoami
root

[bploetz]

@laurentgo regarding building the image as part of the maven build and using the artifacts produced from the build instead of downloading the tar file inside the Dockerfile, can you tell me a little bit about the dremio CI workflow, i.e. is each maven artifact built individually, or is the entire project built as a monolith? I basically need to know how to get the dependencies (the built server code and the scripts) into the Docker image.

Thanks.

[laurentgo]

It's pretty much as stated in the README.md doc: CI builds all modules together using mvn clean install

[sfescape]

"I'll take a crack at changing this to create the image as part of the build based on the artifacts produced by the build, rather than downloading them."

Personally I don't like this idea. I think the notion of an official Dockerfile that folks can read/download/customize for themselves has a ton of merit.