gdalUtils icon indicating copy to clipboard operation
gdalUtils copied to clipboard

Fix --config parameter

Open mwip opened this issue 5 years ago • 3 comments

in gdalUtils::gdal_translate() one can set config parameters using the double-dashed option config which will be forwarded to gdal_translate.

The correct usage of this in gdal would be as a key value pair:

gdal_translate --config GDAL_CACHEMAX "42%"

However, gdalUtils calls it as follows, resulting in an error (I am aware that this would result in an error anyways as no file is supplied, yet it would be different).

gdal_config <- c('GDAL_CACHEMAX "42%"')
guf_aoi <- gdalUtils::gdal_translate(config = gdal_config)

# ERROR 1: --config option given without a key and value argument.
# Warning message:
# In system(cmd, intern = TRUE) :
  running command '"/usr/bin/gdal_translate" -of "GTiff"   --config "GDAL_CACHEMAX "42%""' had status 1

I think the best way to fix this is to use named chr vectors like this:

gdal_config <- c(GDAL_CACHEMAX="30%")

mwip avatar May 14 '20 10:05 mwip

Same to access /vsis3 files. We need to inform AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in --config parameter and the same error raises.

rolfsimoes avatar May 15 '20 01:05 rolfsimoes

@jgrn307 these parameters are kind of a combination between parameter_noquotes and parameter_doubledash. How should these be handeled? Maybe as a new category parameter_config or parameter_noquote_doubledash?

Let me know and I'd be happy to work on the PR.

mwip avatar May 17 '20 20:05 mwip

I think --config could be a parameter_named kind of parameter. Also, it needs to be combined with parameter_doubledash. parameter_named could be provided as a named array, just like @mwip suggested. For example, to access a file in AWS bucket, I need to provide the parameters AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (and sometimes AWS_REQUEST_PAYER. Then, I should write:

gdalinfo('/vsis3/MyBucketName/MyFileKeyId.tif', 
	config = c(AWS_ACCESS_KEY_ID='MyAccessKeyIdString',
		AWS_SECRET_ACCESS_KEY='MySecretAccessKeyString')

This should generate the following command:

gdalinfo /vsis3/MyBucketName/MyFileKeyId.tif --config AWS_ACCESS_KEY_ID "MyAccessKeyIdString" --config AWS_SECRET_ACCESS_KEY "MySecretAccessKeyString"

The implementation I suggest could be:

  • Change gdal_cmd_builder function adding a new parameter parameter_named, as showed
gdal_cmd_builder <- function(executable,parameter_variables=c(),
		parameter_values=c(), parameter_order=c(), parameter_noflags=c(),
		parameter_doubledash=c(),
		parameter_noquotes=c(),
		parameter_named=c(),  # <-- here
		gdal_installation_id=1,
		python_util=FALSE,
		verbose=FALSE)
  • All functions that use config need inform it as parameter_double_dash and parameter_named;
  • In gdal_cmd_builder function body, you could substitute the code
parameter_variables_character_string <- paste(flag,
	qm(parameter_values[[which(names(parameter_values)==X)]]), sep="")

by

parameter_values_actual <- parameter_values[[which(names(parameter_values) == X)]]
if (X %in% parameter_named && !is.null(names(parameter_values_actual))) 
{
	parameter_variables_character_string <- 
		paste(paste(flag, mapply(paste, names(parameter_values_actual),
			qm(parameter_values_actual)), sep = " "), collapse = " ")
} else 
{
	parameter_variables_character_string <- 
		paste(paste(flag, qm(parameter_values_actual), sep = " "), collapse = " ")
}

For completeness, this same code could be inserted in other code sections of the gdal_cmd_builder body to provide all possible combinations of parameter types.

rolfsimoes avatar May 17 '20 22:05 rolfsimoes