nerd
nerd copied to clipboard
Prevent jobs from running while a dataset is still being downloaded or uploaded
Instead of first uploading it makes sense to first create a dataset when metadata of a dataset will be stored in the Kubernetes custom resources. In this case it might be possible for the flex volume of a pod to start pulling the dataset before it is fully uploaded. This can be countered by keeping the phase of the dataset as metadata in the custom resource.