sf
sf copied to clipboard
r-spatial
z error when reading geojson data
I'm trying to read the nytimes' elections dataset by using st_read. After downloading and extracting the file, I get the following error (sf version 0.9-7):
> data <- read_sf("~Downloads/precincts-with-results.geojson")
Error in CPL_get_z_range(obj, 3) : z error - expecting three columns;
System details
platform x86_64-pc-linux-gnuarch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 6.3
year 2020
month 02
day 29
svn rev 77875
language R
version.string R version 3.6.3 (2020-02-29)
nickname Holding the Windsock
sessionInfo()
R version 3.6.3 (2020-02-29) Platform: x86_64-pc-linux-gnu (64-bit) Running under: elementary OS 0.4.1 LokiMatrix products: default BLAS: /usr/lib/libblas/libblas.so.3.6.0 LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
locale:
[1] LC_CTYPE=en_US.UTF-8
[2] LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8
[8] LC_NAME=C
[9] LC_ADDRESS=C
[10] LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C
attached base packages: [1] stats graphics grDevices utils datasets [6] methods base
other attached packages:
[1] forcats_0.5.0 stringr_1.4.0 dplyr_1.0.2
[4] purrr_0.3.4 readr_1.4.0 tidyr_1.1.2
[7] tibble_3.0.4 ggplot2_3.3.2 tidyverse_1.3.0
[10] sf_0.9-7
loaded via a namespace (and not attached):
[1] Rcpp_1.0.5 cellranger_1.1.0
[3] pillar_1.4.7 compiler_3.6.3
[5] dbplyr_1.4.4 class_7.3-17
[7] tools_3.6.3 lubridate_1.7.9
[9] jsonlite_1.7.2 lifecycle_0.2.0
[11] gtable_0.3.0 pkgconfig_2.0.3
[13] rlang_0.4.9 reprex_0.3.0
[15] DBI_1.1.1 cli_2.2.0
[17] rstudioapi_0.13 haven_2.3.1
[19] e1071_1.7-4 withr_2.3.0
[21] xml2_1.3.2 httr_1.4.2
[23] fs_1.5.0 hms_0.5.3
[25] generics_0.0.2 vctrs_0.3.6
[27] classInt_0.4-3 grid_3.6.3
[29] tidyselect_1.1.0 glue_1.4.2
[31] R6_2.5.0 fansi_0.4.1
[33] readxl_1.3.1 modelr_0.1.8
[35] blob_1.2.1 magrittr_2.0.1
[37] backports_1.2.0 scales_1.1.1
[39] ellipsis_0.3.1 units_0.6-7
[41] rvest_0.3.6 assertthat_0.2.1
[43] colorspace_2.0-0 KernSmooth_2.23-17
[45] stringi_1.5.3 munsell_0.5.0
[47] broom_0.7.1 crayon_1.3.4
yep, I get this error too. I'll take a look in the morning (Australian time).
In the meantime, this works
data <- geojsonsf::geojson_sf("~/Downloads/precincts-with-results.geojson")
Actually, I've had a quick look, and there are some geometries with a z property
df <- sfheaders::sf_to_df(
sf = sf
)
library(data.table)
setDT( df )
df[ !is.na( z ), ]
sfg_id multipolygon_id polygon_id linestring_id x y z
1: 33209 NA 33209 1 -95.42749 37.96577 0
2: 33209 NA 33209 1 -95.42737 37.96577 0
3: 33209 NA 33209 1 -95.42738 37.96657 0
4: 33209 NA 33209 1 -95.42740 37.96932 0
5: 33209 NA 33209 1 -95.42740 37.97120 0
---
933443: 37279 NA 37279 1 -97.37149 37.56692 0
933444: 37279 NA 37279 1 -97.37200 37.56692 0
933445: 37279 NA 37279 1 -97.37199 37.56791 0
933446: 37279 NA 37279 1 -97.37149 37.56791 0
933447: 37279 NA 37279 1 -97.37149 37.56750 0
But they're all 0
> df[ !is.na( z ) & z > 0 ]
Empty data.table (0 rows and 7 cols): sfg_id,multipolygon_id,polygon_id,linestring_id,x,y...
And here is a minimally reproducible example
sf::st_sfc(
list(
sf::st_point(c(0,0))
, sf::st_point(c(1,1,1))
)
)
Error in CPL_get_z_range(unclass(obj)[sel], 0) :
z error - expecting three coordinates
@edzer
With this example
sf::st_sfc(
list(
sf::st_point(c(0,0))
, sf::st_point(c(1,1,1))
)
)
If I let it pass through the CPL_get_z_range() C++ functions, it ends up hitting this stop point
# set n_empty, check XY* is uniform:
if (is.null(attr(lst, "n_empty")) || any(is_null)) { # n_empty is set by CPL_read_wkb:
attr(lst, "n_empty") = sum(vapply(lst, sfg_is_empty, TRUE))
if (length(u <- unique(sfg_classes[1L,])) > 1)
stop(paste("found multiple dimensions:", paste(u, collapse = " ")))
}
For examle
sf::st_sfc(
list(
sf::st_point(c(0,0))
, sf::st_point(c(1,1,1))
)
)
Error in sf::st_sfc(list(sf::st_point(c(0, 0)), sf::st_point(c(1, 1, 1)))) :
found multiple dimensions: XY XYZ
But the read_sf() example will work
data <- read_sf("~/Downloads/precincts-with-results.geojson")
nrow(data)
[1] 109363
Is it perhaps best to allow multiple dimensions, given the various st_ functions can handle them?
For example
geo <- '
{
"type":"FeatureCollection",
"features": [
{
"type":"Feature",
"geometry": {
"type": "LineString",
"coordinates": [[0,0],[1,1]]
}
},
{
"type":"Feature",
"geometry": {
"type": "LineString",
"coordinates": [[0,0,0],[1,1,0]]
}
}
]
}'
sf <- st_read(geo)
sf::st_length( sf )
Units: [m]
[1] 156899.6 110574.4
(The next thing for me to understand is why adding in the Z coordinate here creates a differnet 'length').
Thanks for looking into this! I'd like to see st_read succeed in such cases, best would be with a warning and a suggestion how to clean up the mess.
I remember running into a similar issue in the past and I solved it with st_zm (which is obviously not possible with st_read.)
And thanks for the amazing work you guys are doing!
Hi, yeah, sorry - got sidetracked with something I'm working on.
I think a limiting factor is the current requirement to have the dimension uniform for all geometries.
How do you think this should be handled?
The purist in me would say: leave this an error, let the user solve the problem, and let the user try again after solving it. But then: how should she solve it? Before we computed z & m ranges (and generated errors), an object was returned with mixed dimensions, raised a warning, and pointed to st_zm() to correct this (remove all z, m, or zm geometries). Would it be doable to go back to that state?
@dcooley the requirement you refer to is skipped when we come back from CPL_read_wkb, i.e. when we read data from file.
Yes, for example this works
geo <- '{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"id":1.0},"geometry":{"type":"Point","coordinates":[1.0,2.0]}},{"type":"Feature","properties":{"id":1.0},"geometry":{"type":"LineString","coordinates":[[1.0,2.0,3.0,4.0]]}}]}'
sf <- sf::st_read( geo )
sf
# Simple feature collection with 2 features and 1 field
# geometry type: GEOMETRY
# dimension: XY
# bbox: xmin: 1 ymin: 2 xmax: 1 ymax: 2
# z_range: zmin: 3 zmax: 3
# geographic CRS: WGS 84
# id geometry
# 1 1 POINT (1 2)
# 2 1 LINESTRING Z (1 2 3)
However, the dimension attributed is printed as XY, even though there are mixed dimensions.
Is it worth considering removing this uniform check, and perhaps printing the different dimensions that exist, something like
sf
# Simple feature collection with 2 features and 1 field
# geometry type: GEOMETRY
# dimension: XY, XYZ
# bbox: xmin: 1 ymin: 2 xmax: 1 ymax: 2
# z_range: zmin: 3 zmax: 3
# geographic CRS: WGS 84
# id geometry
# 1 1 POINT (1 2)
# 2 1 LINESTRING Z (1 2 3)
I think we also need to let it pass the CPL_get_z_range() function to avoid the original error
Error in CPL_get_z_range(obj, 3) : z error - expecting three columns;
Yes. Return NA?
On March 7, 2021 3:50:02 AM GMT+01:00, Dave [email protected] wrote:
I think we also need to let it pass the
CPL_get_z_range()function to avoid the original errorError in CPL_get_z_range(obj, 3) : z error - expecting three columns;-- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: https://github.com/r-spatial/sf/issues/1592#issuecomment-792180468
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Saw your comment .
I'm actually struggling to compile sf on my M1 Mac. Are there any open issues I can read through to get it working?
FYI: this error occurs in sf v1.0-7.
The way-around suggested by @dcooley (i.e. using geojsonsf::geojson_sf) worked fine. Thanks for the hint and for the great work that you have been doing!
I'm seeing this error from the JSON file downloaded from: https://149613070.v2.pressablecdn.com/coordinates/indigenousTerritories.json (4Mb) using 1.0-8:
> library(sf)
Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1; sf_use_s2() is TRUE
> terr = st_read("./indigenousTerritories.json")
Reading layer `indigenousTerritories' from data source
`/nobackup/rowlings/Downloads/SO/indigenousTerritories.json'
using driver `GeoJSON'
Error in CPL_get_z_range(obj, 3) : z error - expecting three columns;
> packageVersion("sf")
[1] ‘1.0.8’
Current fix is to use ogr2ogr to flatten it all to XY using the -dim option, but I'm also wondering why passing the right type= code to st_read has no effect.
Is this issue still open? One of my students is running into the same issue loading a regular (ESRI) shapefile (latest R version 4.2.2, sf version 1.0.9 and rgdal version 1.6.2). The code worked last year (probably an older sf version). We used this (https://github.com/r-spatial/sf/issues/2046#issuecomment-1326351674) solution to enable her to read in her shapefile again and re-run her analysis and map. But I would prefer to have an actual fix...
Eager for a fix for this. This backwards incompatible change in what should be a patch release of 1.0.9 has made life difficult for us, and we have having to downgrade manually to st 1.0.8 - this kind of change should have been introduced in a 2.0 release. Would also be happy with a workaround that didn't involve a deprecated library such as rgdal.
When using rgdal::readOGR I do indeed receive the message "Z-dimension discarded" - the shapefile in question can be found at https://github.com/IMERSS/maxwell/tree/main/spatial_data/vectors/Shp_files/Watershed_CRD and was exported from QGIS.
I still get the same error on sf_1.0-9, rgdal_1.6-3 Interestingly it works with a polygon layer with a z range, but not with a multi-polygon feature layer. any suggestions?
@abdulrr, make sure you updated {sf} to the devel version (1.0.10).
install.packages("sf", repos = "https://r-spatial.r-universe.dev")
or
remotes::install_github("r-spatial/sf")
Hi, I still have this problem using 1.0-9 and cannot manage to install 1.10 using either of the two commands mentioned above. Would anyone have a solution?
Hi, I still have this problem using 1.0-9 and cannot manage to install 1.10 using either of the two commands mentioned above. Would anyone have a solution?
@ItsMeVictor, I think it will be better if you open a new issue and post the installation log with error and session information.
Confirmed: I just hit this issue and tried the work-around.
geojsonsf::geojson_sf("file.geojson)
worked for me
