raster-vision icon indicating copy to clipboard operation
raster-vision copied to clipboard

REQUEST: rastervision check command

Open nanotubing opened this issue 3 years ago • 1 comments

one difficulty I've run into so far is that if there is an invalid S3 URL fed to rastervision, you tend to not find out until the job has already been running for some time. It may be useful to be able to run a rastervision check command, which would verify all URLs, and report any that are not accessible.

rastervision run local -a file1 file.txt ... check

nanotubing avatar Apr 29 '21 19:04 nanotubing

It should be pretty straightforward to write a function that takes in a DatasetConfig object and checks if the input files exist. Then you could call that in your get_config function. There is a file_exists function that would be useful here: https://github.com/azavea/raster-vision/blob/master/rastervision_pipeline/rastervision/pipeline/file_system/utils.py#L184 It will be difficult to write that function in a very generic way, but it should be easy to write it if you know the types of RasterSource and LabelSource you will be using.

We actually used to have support for doing this automatically in an earlier version of RV, but it seemed to add too much complexity to the code, so I got rid of it. Maybe that was a bad idea :)

lewfish avatar May 14 '21 16:05 lewfish