jetstream
jetstream copied to clipboard
Optimize/reduce calls to Experimenter API
Currently, each analysis run for each experiment and analysis date calls the Experimenter API to fetch all experiments. With 50 runs or more running in parallel which is more requests than the API can handle. There are a few options we could look into:
- passing the configs into the Argo containers. A few limitations in the past prevented us from doing this, but perhaps with the new jetstream-conifg-parser we could take another look
- alternatively instead of making a call to get all experiments, we could make the call to
/experiments/<slug>/
for specific experiments
┆Issue is synchronized with this Jira Task