r-bridge icon indicating copy to clipboard operation
r-bridge copied to clipboard

POSIXct class assigned incorrectly

Open Nova-Scotia opened this issue 1 year ago • 5 comments

Hi there,

When I import data from ArcGIS Online and check the class of a date-time column, the column classes are being listed as c("POSIXt", "POSIXct"). But shouldn't they be listed in order of most specific to least specific (so, reversed)? This is causing problems for me downstream when I try to join these data to another dataframe because it sees them as incompatible types. @JosiahParry same is happening with arcgislayers....

My other dataframes have the class orders reversed. For more details see https://stackoverflow.com/questions/77247242/change-order-of-classes-in-column

deployments_raw <- arc.select(arc.open("https://services1.arcgis.com/TJgobll56655ydfgo/arcgis/rest/services/Deployments/FeatureServer/0"))
dput(deployments_raw$deploy_on_date[1])
structure(1667599200, class = c("POSIXt", "POSIXct"))

Nova-Scotia avatar Oct 06 '23 20:10 Nova-Scotia

That's very odd. I do not know what happens with arcgisbinding, but arcgislayers uses as.POSIXct() to create the date time field. Perhaps there is somewhere else upstream where your class is being modified? I will create a feature service with date time to test with.

https://github.com/R-ArcGIS/arcgisutils/blob/494e17003c3fa5ca08ab2b5b9365126f01b15701/R/utils-dates.R#L42C1-L42C1

In the mean time here is a simple function to fix your classes with:

x <-  Sys.time()

class(x) <- rev(class(x))

restore_posixct <- function(x) {
  if (identical(class(x), c("POSIXt", "POSIXct"))) {
    class(x) <- rev(class(x))
  }
  x
}

class(x)
#> [1] "POSIXt"  "POSIXct"
class(restore_posixct(x))
#> [1] "POSIXct" "POSIXt"

Note that as.POSIXct() will not reorder the class attributes nor will lubridate::as_datetime() which leads me to think this might be a bug in dplyr or vctrs where the class order is being checked perhaps too strictly.

JosiahParry avatar Oct 06 '23 21:10 JosiahParry

In the case of {arcgislayers}, the class is assigned correctly. Will report back with the result of arcgisbinding.

library(arcgis)
#> Attaching core arcgis packages:
#>   - {arcgisutils} v0.1.0
#>   - {arcgislayers} v0.1.0

url <- "https://services1.arcgis.com/vkTwD8kHw2woKBqV/arcgis/rest/services/ESCAD_Current_Incidents_Public/FeatureServer/0"
arc_open(url) |> 
  arc_select() -> tmp

class(tmp$Response_Date)
#> [1] "POSIXct" "POSIXt"

JosiahParry avatar Oct 06 '23 21:10 JosiahParry

Confirmed that this is a bug with {arcgisbinding} CC @dpavlushko

library(arcgisbinding)
#> *** Please call arc.check_product() to define a desktop license.
arc.check_product()
arc.check_portal()

url <- "https://services1.arcgis.com/vkTwD8kHw2woKBqV/arcgis/rest/services/ESCAD_Current_Incidents_Public/FeatureServer/0"

tmp <- arc.open(url) |> 
  arc.select()

class(tmp$Response_Date)
#> [1] "POSIXt"  "POSIXct"

JosiahParry avatar Oct 06 '23 21:10 JosiahParry

Fortunately a super simple fix that I'll try and address next week. It's just one line that needs to be changed. @Nova-Scotia, otherwise you can use arcgislayers to get the class assignment correct, or you can use the helper function above in the mean time. Sorry about this!

JosiahParry avatar Oct 06 '23 21:10 JosiahParry

Thank you so much @JosiahParry! It took me longer than I'd like to admit to figure out what was happening... as usual making a minimal reproducible example helped me ferret out the issue.

Nova-Scotia avatar Oct 11 '23 01:10 Nova-Scotia