zenml icon indicating copy to clipboard operation
zenml copied to clipboard

Scalability improvements: Reduce misc/hydration server requests

Open schustmi opened this issue 1 year ago • 0 comments

Describe changes

This PR contains some improvements that reduce the amount of server requests that we send when running a step

  • All SQLZenStore methods now correctly include the resources of a model
  • The active workspace is only fetched once
  • The active stack model is only fetched once
  • Hydrating a response model now sets both the resources and metadata, which before would require two separate requests
  • When trying to fetch the model version for a step: Instead of sending one hydration request per already executed step of the pipeline run, we now send a single request instead
  • Only register a new active filesystem if it doesn't match the active artifact store
  • Add flavor to ComponentResponseResources

Breaking changes (still requires frontend changes):

  • Move ArtifactVersionResponse.artifact_store_id from metadata to body
  • Move FlavorResponse.source from metadata to body
  • Rename ComponentResponse.flavor to ComponentResponse.flavor_name

Pre-requisites

Please ensure you have done the following:

  • [ ] I have read the CONTRIBUTING.md document.
  • [ ] If my change requires a change to docs, I have updated the documentation accordingly.
  • [ ] I have added tests to cover my changes.
  • [ ] I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
  • [ ] If my changes require changes to the dashboard, these changes are communicated/requested.

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Other (add details above)

schustmi avatar Oct 17 '24 09:10 schustmi