edx-analytics-pipeline icon indicating copy to clipboard operation
edx-analytics-pipeline copied to clipboard

Support a dockerized analyticstack

Open mulby opened this issue 9 years ago • 4 comments

This PR is part of the dockerization of the analyticstack, most of the documentation can be found here: https://github.com/edx/configuration/pull/3582

It implements the following changes:

  1. Add a docker.cfg config file that contains all of the settings needed to run in a docker-compose environment.
  2. Add Makefile targets specified by OEP-5 that allow a standard interface for managing the containers.
  3. Support running acceptance tests with "launch-task" instead of "remote-task". This allows us to run the acceptance tests from the container without SSH-ing into itself and re-deploying the pipeline when executing every test. Eventually this could be used to run the acceptance tests in Travis or another third-party CI system that supports docker-compose based test execution.
  4. A bunch of detail in the README.md describing how to set it all up.

FYI: @HassanJaveed84

TODO:

  • [x] Implement the OEP-5 standard Makefile targets
  • [x] Clean up the formatting of README.md
  • [ ] Implement the part of OEP-5 that specifies how containers are built

mulby avatar Dec 28 '16 14:12 mulby

Current coverage is 73.79% (diff: 100%)

Merging #337 into master will decrease coverage by 5.94%

@@             master       #337   diff @@
==========================================
  Files           191         92     -99   
  Lines         19676      10236   -9440   
  Methods           0          0           
  Messages          0          0           
  Branches          0          0           
==========================================
- Hits          15691       7554   -8137   
+ Misses         3985       2682   -1303   
  Partials          0          0           

Powered by Codecov. Last update 33eec79...6524be2

codecov-io avatar Dec 29 '16 14:12 codecov-io

I'm sure you've seen this, but it looks like we need to open up those luigid images, or add them if they're actually just missing.

tobz avatar Jan 10 '17 14:01 tobz

@tobz yeah none of the images have been pushed up to docker hub yet... so the docker-compose stuff won't yet work in Travis. I was mostly just sketching out how it could work.

mulby avatar Jan 10 '17 14:01 mulby

A ha! Gotcha. :)

tobz avatar Jan 10 '17 14:01 tobz