raster-vision
raster-vision copied to clipboard
REQUEST: rastervision check command
one difficulty I've run into so far is that if there is an invalid S3 URL fed to rastervision, you tend to not find out until the job has already been running for some time. It may be useful to be able to run a rastervision check command, which would verify all URLs, and report any that are not accessible.
rastervision run local -a file1 file.txt ... check
It should be pretty straightforward to write a function that takes in a DatasetConfig
object and checks if the input files exist. Then you could call that in your get_config
function. There is a file_exists
function that would be useful here: https://github.com/azavea/raster-vision/blob/master/rastervision_pipeline/rastervision/pipeline/file_system/utils.py#L184 It will be difficult to write that function in a very generic way, but it should be easy to write it if you know the types of RasterSource
and LabelSource
you will be using.
We actually used to have support for doing this automatically in an earlier version of RV, but it seemed to add too much complexity to the code, so I got rid of it. Maybe that was a bad idea :)