experiments
experiments copied to clipboard
IntelAI
Experiments API for Experiment Tracking on Kubernetes
To support the broadest potential user base we should ensure that the API installs and runs under python2 not just python3. This will require at least the following: - building...
Version numbers are not specified for dependencies in `requirements.txt` which prevents repeatable builds. At a bare minimum we should pin these, though it may instead be worth switching to: [pipenv](https://docs.pipenv.org/)...
Typical python package practice is to import relative to some top-level namespace and to put all the modules under a subdir of the same name. Right now this project has...
``` metadata.name: Invalid value: \"hierarchical_goals\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex...
As an optimizer author, I would like some abstraction to help me interpret the status of jobs and for failure, a higher level indication of _why_ it failed. Say, OOM,...
The call to `load_kube_config` should allow the user to specify a `context` variable: https://github.com/IntelAI/experiments/blob/master/lib/__init__.py#L9. This could be done through a sensibly named environment variable.
I'm not sure the best way to do this, but it would be nice to be able to easily keep the name of the job and the name of the...
This might not be the best interface/name, but it would be nice to have a simple method for checking to see if an experiment already exists and adding to it...
For example: ``` client.create_job(experiment, { 'master': { 'noise_eps': 0.5, }, 'sub': { 'noise_eps': 0.1, }, }) ``` These could be serialized as: `MASTER__NOISE_EPS` with double under: `__` separating levels.
Doesn't work: ``` experiment = exp.Experiment(name=name, job_template=job['spec']) client.create_experiment(experiment) job = client.create_job(experiment, parameters) ``` Does: ``` experiment = exp.Experiment(name=name, job_template=job['spec']) experiment = client.create_experiment(experiment) job = client.create_job(experiment, parameters) ``` The error message...