jetstream icon indicating copy to clipboard operation
jetstream copied to clipboard

Optimize/reduce calls to Experimenter API

Open scholtzan opened this issue 2 years ago • 1 comments

Currently, each analysis run for each experiment and analysis date calls the Experimenter API to fetch all experiments. With 50 runs or more running in parallel which is more requests than the API can handle. There are a few options we could look into:

  • passing the configs into the Argo containers. A few limitations in the past prevented us from doing this, but perhaps with the new jetstream-conifg-parser we could take another look
  • alternatively instead of making a call to get all experiments, we could make the call to /experiments/<slug>/ for specific experiments

┆Issue is synchronized with this Jira Task

scholtzan avatar Aug 08 '22 17:08 scholtzan