prefect
prefect copied to clipboard
Sparse checkout for git pulls
Describe the current behavior
When cloning a git repository, the entire repository is cloned, this can take longer than the flow run itself.
Describe the proposed behavior
Additional configuration in prefect.yaml, and equivalent commands should permit a sparse checkout, maybe just of a single directory.
An additional touch would be to make the selected directory become the base of the clone, rather than retaining the full path, making it easy to just keep paths relative in code.
Example Use
in prefect.yaml
pull:
- prefect.deployments.steps.git_clone:
repository: https://github.com/org/repo.git
access_token: '{{ prefect.blocks.secret.github-token }}'
directory: that/one/directory/where/my/code/is
Additional context
No response