determined
determined copied to clipboard
perf: resolve get_experiment_checkpoints slowness
Description
current get checkpoints for experiment does
SELECT *
FROM proto_checkpoints_view c
WHERE CAST(c.training->>'experiment_id' AS integer) = $1
ORDER BY c.report_time DESC
proto_checkpoints_view
looks like
SELECT c.uuid::text AS uuid,
...
jsonb_build_object('trial_id', c.trial_id, 'experiment_id' ... )
FROM checkpoints_view c;
burying the experiment_id
within the json_build_object
prevents query planner from filtering by experiment_id
before doing the join here:
FROM checkpoints_v2 c
LEFT JOIN trials t ON c.task_id = t.task_id
LEFT JOIN experiments e ON t.experiment_id = e.id
resulting in scan of checkpoints.
adding c.experiment_id
to proto_checkpoints_view
and changing get_checkpoints_for_experiment.sql
to
SELECT *
FROM proto_checkpoints_view c
WHERE c.experiment_id = $1
ORDER BY c.report_time DESC;
resolves the query performance issues but breaks the proto conversion with
"error fetching checkpoints for experiment 48 from database: error running query: get_checkpoints_for_experiment: error converting row to Protobuf struct: proto: (line 1:62): unknown field \"experiment_id\""
(also breaks the other APIs that use this view).
Test Plan
gcloud sql connect determined-release-party
run the queries.
Commentary (optional)
further discussion in code comments
Checklist
- [ ] Changes have been manually QA'd
- [ ] User-facing API changes need the "User-facing API Change" label.
- [ ] Release notes should be added as a separate file under
docs/release-notes/
. See Release Note for details. - [ ] Licenses should be included for new code which was copied and/or modified from any external code.
- [ ] If modifying
/webui/react/src/shared/
verifymake -C webui/react test-shared
passes.
Deploy Preview for storybook-det canceled.
Name | Link |
---|---|
Latest commit | 350b9689f93f3f603dc1b1c3add3820b1170e81c |
Latest deploy log | https://app.netlify.com/sites/storybook-det/deploys/633b57101a8dec00097349d3 |
Deploy Preview for determined-ui canceled.
Name | Link |
---|---|
Latest commit | 350b9689f93f3f603dc1b1c3add3820b1170e81c |
Latest deploy log | https://app.netlify.com/sites/determined-ui/deploys/633b571098598b00089c10e8 |