skyplane
skyplane copied to clipboard
[CLI] Implement `--include` and `--exclude` to filter object prefixes
Although Skyplane can transfer bucket folders using the -r
flag, it cannot match prefixes with regex to transfer subsets of a folder (e.g. s3://bucket/imagenet/train_*
).
The AWS CLI supports filtering based off simple regex expression https://docs.aws.amazon.com/cli/latest/reference/s3/#use-of-exclude-and-include-filters. We should add similar functionality to the Skyplane CLI, since this is probably a common use-case.