prefect icon indicating copy to clipboard operation
prefect copied to clipboard

Sparse checkout for git pulls

Open cpinflux opened this issue 5 months ago • 2 comments

Describe the current behavior

When cloning a git repository, the entire repository is cloned, this can take longer than the flow run itself.

Describe the proposed behavior

Additional configuration in prefect.yaml, and equivalent commands should permit a sparse checkout, maybe just of a single directory.

An additional touch would be to make the selected directory become the base of the clone, rather than retaining the full path, making it easy to just keep paths relative in code.

Example Use

in prefect.yaml

pull:
- prefect.deployments.steps.git_clone:
    repository: https://github.com/org/repo.git
    access_token: '{{ prefect.blocks.secret.github-token }}'
    directory: that/one/directory/where/my/code/is

Additional context

No response

cpinflux avatar Sep 03 '24 14:09 cpinflux