Cook
Cook copied to clipboard
twosigma
Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark
For the caches added in https://github.com/twosigma/Cook/pull/731 report metrics on the caches hit/miss rates
For example: ```bash $ cd $TEST_DIR && $TEST_CMD Python 3.6.3 running nosetests running egg_info writing cook_executor.egg-info/PKG-INFO writing dependency_links to cook_executor.egg-info/dependency_links.txt writing entry points to cook_executor.egg-info/entry_points.txt writing requirements to cook_executor.egg-info/requires.txt writing...
Almost all of our integration tests are in a single file: test_basic.py. Breaking up the file would make it easier to find related tests, and hopefully help us avoid the...
We're currently caching the UUIDs of our jobs/instances/groups in the Liberator context when processing the decision handlers for our REST endpoints. That choice results in a lot of repeated UUID→Entity...
We should move (and probably rename) the top-level *simulator* directory. It probably makes more sense to include it under the *integration* directory, and call it something like *workloads*.
#582 deprecates the `/rawscheduler` endpoint in favor of new entity endpoints. We should be using `/jobs`, `/instances` and `/groups` consistently in all of our tools (CLI, simulator, etc) and all...
``` ====================================================================== ERROR: test_docker_port_mapping (tests.cook.test_basic.CookTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/twosigma/Cook/integration/tests/cook/test_basic.py", line 1149, in test_docker_port_mapping job = util.wait_for_job(self.cook_url, job_uuid, 'running') File "/home/travis/build/twosigma/Cook/integration/tests/cook/util.py", line 308, in wait_for_job return wait_for_jobs(cook_url,...
We've been seeing repeated failure of this test. I think Paul has already determined that this is due to some broken logic in the constraints handling code. ``` ====================================================================== ERROR:...
@wyegelwel Not sure how easy this would be to add (if it is not easy, just create a to-do or issue) to track a metric of how long it takes...