timeseriesdb icon indicating copy to clipboard operation
timeseriesdb copied to clipboard

Default for valid_on breaks on more recent version of dependencies

Open HomoCodens opened this issue 3 years ago • 3 comments

It looks like either R 4.x or RPostgres 1.4.x translates NA to a boolean instead of NULL, which breaks this default for valid_on:

  # RPostgres plays nicer with NA than with NULL
  if(is.null(valid_on)) {
    valid_on <- NA
  }

...

res <- dbSendQuery(con, sprintf("select * from %sts_read_raw(%s, %s)",
                                                     dbQuoteIdentifier(con, Id(schema = schema)),
                                                     dbQuoteLiteral(con, valid_on),
                                                     dbQuoteLiteral(con, respect_release_date)))

It produces

Error: Failed to prepare query: ERROR:  function timeseries.ts_read_raw(boolean, boolean) does not exist
LINE 1: select * from "timeseries".ts_read_raw(NULL::bool, FALSE)
                      ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

There is no ts_read_raw(boolean, boolean), only ts_read_raw(date, boolean) which makes me suspect that the NA is now getting trandlated into a boolean instead of the NULL it used to.

The workaround for now is to just supply valid_on = Sys.Date() instead of leaving it empty.

Sessioninfo where it works:

R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=German_Switzerland.1252  LC_CTYPE=German_Switzerland.1252    LC_MONETARY=German_Switzerland.1252
[4] LC_NUMERIC=C                        LC_TIME=German_Switzerland.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] timeseriesdb_1.0.0-1.1.2

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7        lattice_0.20-38   zoo_1.8-8         grid_3.5.1        jsonlite_1.7.2    DBI_1.1.0        
 [7] rlang_0.4.9       data.table_1.14.2 blob_1.2.1        vctrs_0.3.6       xts_0.12-0        RPostgres_1.2.0  
[13] tools_3.5.1       bit64_0.9-7       bit_1.1-15.2      hms_0.5.3         yaml_2.2.1        compiler_3.5.1   
[19] pkgconfig_2.0.3 

SessionInfo where it does not:

R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_Switzerland.1252 LC_CTYPE=English_Switzerland.1252 LC_MONETARY=English_Switzerland.1252
[4] LC_NUMERIC=C LC_TIME=English_Switzerland.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] timeseriesdb_1.0.0-1.1.2 tsbox_0.3.1 MacrobondAPI_1.2-6 xts_0.12.1 zoo_1.8-9

loaded via a namespace (and not attached):
[1] Rcpp_1.0.7 lubridate_1.8.0 lattice_0.20-45 withr_2.4.3 grid_4.1.2 jsonlite_1.7.2 lifecycle_1.0.1
[8] DBI_1.1.2 rlang_0.4.12 data.table_1.14.2 blob_1.2.2 generics_0.1.1 vctrs_0.3.8 ellipsis_0.3.2
[15] RPostgres_1.4.2 tools_4.1.2 bit64_4.0.5 bit_4.0.4 anytime_0.3.9 hms_1.1.1 compiler_4.1.2
[22] pkgconfig_2.0.3

HomoCodens avatar Dec 23 '21 09:12 HomoCodens

Let's move the check:

  tsl <- db_with_tmp_read(con,
                          keys_unique,
                          regex,
                          {
                            res <- dbSendQuery(con, sprintf("select * from %sts_read_raw(%s, %s)",
                                                     dbQuoteIdentifier(con, Id(schema = schema)),
                                                     if(is.null(valid_on) | is.na(valid_on)) "NULL" else dbQuoteLiteral(con, valid_on),
                                                     dbQuoteLiteral(con, respect_release_date)))
                            tsl <- get_tsl_from_res(res, chunksize)
                            dbClearResult(res)
                            tsl
                          },
                          schema = schema)

HomoCodens avatar Jan 24 '22 15:01 HomoCodens

And while we're at it add a check that valid_on is a date vector of length 1.

HomoCodens avatar Jan 24 '22 15:01 HomoCodens

The problem was that RPostgres 1.4.3 adds type casts in dbQuoteLiteral so dbQuoteLiteral(con, NA) becomes NULL:bool instead of the previous NULL.

HomoCodens avatar Jan 24 '22 15:01 HomoCodens