Improve handling for input URLs with appended queries

[elipousson]

Is your feature request related to a problem? Please describe.

When I copy a GeoService URL from a layer information page on a ArcGIS Hub site, it always has a default query appended "/query?outFields=*&where=1%3D1". The URL returns an error that isn't even very helpful at figuring out what went wrong.

Describe the solution you'd like

If possible, I'd like to see arcgislayers support queries passed as part of the URL. This would also be handy for setting up filtered views interactively with a non-R user and then reusing the generated URL rather than needing to reconstruct the query separately.

Either way, it may be helpful to add appropriate warning or error messages for URLs an appended query to avoid confusion by users.

This reprex illustrates a basic potential solution that could be incorporated into arc_select():

library(arcgislayers)

url <- "https://geodata.md.gov/imap/rest/services/Boundaries/MD_PhysicalBoundaries/FeatureServer/1/query?outFields=*&where=1%3D1"

data <- arc_read(url)
#> Error in `arc_read()`:
#> ! `url` is not a supported type: "FeatureLayer", "Table", or
#>   "ImageServer"
#> ℹ found "FeatureServer"

parsed_url <- httr2::url_parse(url)

if (!is.null(parsed_url$query)) {
  outFields <- parsed_url$query$outFields
  where <- parsed_url$query$where
  
  parsed_url$query <- NULL
  parsed_url$path <- sub("/query$", "", parsed_url$path)
}

arc_read(
  url = paste0(parsed_url$scheme, "://", parsed_url$hostname, parsed_url$path),
  fields = outFields,
  where = where
)
#> Simple feature collection with 24 features and 7 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: -8848486 ymin: 4568691 xmax: -8354440 ymax: 4825753
#> Projected CRS: WGS 84 / Pseudo-Mercator
#> First 10 features:
#>    OBJECTID         county district county_fip countynum creation_d last_updat
#> 1         1       Allegany        6          1         1 2007-08-08 2012-01-20
#> 2         2   Anne Arundel        5          3         2 2007-01-09 2008-07-01
#> 3         3      Baltimore        4          5         3 2009-11-17 2012-02-15
#> 4         4 Baltimore City        0        510        24 2006-04-18 2009-11-16
#> 5         5        Calvert        5          9         4 2007-01-09 2008-07-08
#> 6         6       Caroline        2         11         5 2007-05-21 2010-01-28
#> 7         7        Carroll        7         13         6 2008-06-16 2012-01-17
#> 8         8          Cecil        2         15         7 2006-04-18 2008-08-20
#> 9         9        Charles        5         17         8 2006-04-18 2010-06-21
#> 10       10     Dorchester        1         19         9 2006-04-18 2007-02-22
#>                          geometry
#> 1  MULTIPOLYGON (((-8725750 48...
#> 2  MULTIPOLYGON (((-8520054 46...
#> 3  MULTIPOLYGON (((-8510314 47...
#> 4  MULTIPOLYGON (((-8519244 47...
#> 5  MULTIPOLYGON (((-8519273 46...
#> 6  MULTIPOLYGON (((-8432189 47...
#> 7  MULTIPOLYGON (((-8564637 47...
#> 8  MULTIPOLYGON (((-8443421 48...
#> 9  MULTIPOLYGON (((-8586624 46...
#> 10 MULTIPOLYGON (((-8439760 46...

^{Created on 2024-08-06 with reprex v2.1.0}

Describe alternatives you've considered

This may be related to the URL validation proposed for arcgisutils so could be incorporated into that package instead: https://github.com/R-ArcGIS/arcgisutils/issues/31

[JosiahParry]

I really like the use of httr2::url_parse()

[JosiahParry]

@elipousson what do you think about something along the lines of this?

# get the url
url <- "https://geodata.md.gov/imap/rest/services/Boundaries/MD_PhysicalBoundaries/FeatureServer/1/query?outFields=*&where=1%3D1"

# parse the url
parsed_url <- httr2::url_parse(url)

# enumerate known path types
known_content_types <- c("FeatureServer", "MapServer", "ImageServer", "GeocodeServer")

# make regex
content_pattern <- paste(known_content_types, collapse = "|")

# extrect kind from URL
url_type <- stringr::str_extract(parsed_url$path, content_pattern)
url_type
#> [1] "FeatureServer"

^{Created on 2024-08-06 with reprex v2.1.0}

[Robinlovelace]

I agree with this and hit something similar:

pak::pkg_install("elipousson/esri2sf")
remotes::install_github("r-arcgis/arcgis", dependencies = TRUE)
res = esri2sf::esrisearch("Local Authority Districts December 2024 Boundaries UK BFE")
u_from_res = paste0(res$url[1], "/0")
library(arcgis)
res_sf = arc_read(u_from_res)
plot(res_sf$geometry)

On a different but related note, are there any plans for search functionality?

[JosiahParry]

@Robinlovelace I'm still not sure how we would handle this. Take the following feature server for example:

https://services3.arcgis.com/ZvidGQkLaDJxRSJ2/arcgis/rest/services/PLACES_LocalData_for_BetterHealth/FeatureServer

appending /0, /1, /2, /3, and /4 are all valid items to be queried.

Search is definitely on the road map. I'll make a comment in the existing issue (#22)

[Robinlovelace]

Could it do what GDAL does when reading multi-layered .gpkg files and read the first layer while emitting a message stating that there are multiple layers?

[Robinlovelace]

Or could the error state that there are multiple layers?

[JosiahParry]

Related https://github.com/R-ArcGIS/arcgisutils/issues/31

I think that having URL helpers would make this a whole lot simpler. In the case of arc_read() one could check if it is a FeatureServer or MapServer if so, cli::cli_alert_warning() and then append /0.

When there is appended queries @elipousson has already added functionality to clear them out

[Robinlovelace]

👍

[JosiahParry]

This functionality is now supported with thanks to @elipousson