osmextract
osmextract copied to clipboard
ropensci
[FEATURE] Get OSM objects by node/way id numbers
Is your feature request related to a problem? Please describe.
I would like to know if there is an osmextract variant for osmdata::opq_osm_id
Describe the solution you'd like I have a large number of OSM way ids for which I would like to obtain the spatial objects. I already downloaded the OSM data for the region of interest.
Describe alternatives you've considered
osmdata::opq_osm_id gives me a time-out error
Additional context None
Hi @mtennekes! If you know that those ways lay in a given OSM extract, I think that you can run something like this:
# packages
library(osmextract)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright.
#> Check the package website, https://docs.ropensci.org/osmextract/, for more details.
# define 5 osm_ids for 5 ways in the Isle of Wight
my_osm_id <- c("123588694", "533585723", "61240667", "570495917", "566733078")
# read + subset those ways
iow_subset <- oe_get(
place = "Isle of Wight",
query = paste0("SELECT * FROM lines WHERE osm_id IN (", paste0(my_osm_id, collapse = ","), ")")
)
#> The input place was matched with: Isle of Wight
#> The chosen file was already detected in the download directory. Skip downloading.
#> The corresponding gpkg file was already detected. Skip vectortranslate operations.
#> Reading query `SELECT * FROM lines WHERE osm_id IN (123588694,533585723,61240667,570495917,566733078)' from data source `C:\Users\andrea.gilardi\Documents\osm-data\geofabrik_isle-of-wight-latest.gpkg'
#> using driver `GPKG'
#> Simple feature collection with 5 features and 9 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: -1.495138 ymin: 50.65387 xmax: -1.143836 ymax: 50.74988
#> Geodetic CRS: WGS 84
Created on 2022-03-08 by the reprex package (v2.0.1)
If you have already downloaded the OSM data using any other method, check oe_read(). Hope that helps.
Thanks! Not successful yet.
This is what I got with osmdata (to compare with):
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
library(osmdata)
dat1 <- opq_osm_id (type = "way", id = ids) %>%
opq_string () %>%
osmdata_sf ()
dat1
which returns:
Object of class 'osmdata' with:
$bbox :
$overpass_call : The call submitted to the overpass API
$meta : metadata including timestamp and version numbers
$osm_points : 'sf' Simple Features Collection with 57 points
$osm_lines : 'sf' Simple Features Collection with 10 linestrings
$osm_polygons : 'sf' Simple Features Collection with 0 polygons
$osm_multilines : NULL
$osm_multipolygons : NULL
So all 10 linestrings are there.
With your method:
iow_subset <- oe_get(
place = "Netherlands",
query = paste0("SELECT * FROM lines WHERE osm_id IN (", paste0(ids, collapse = ","), ")")
)
iow_subset
with output
The input place was matched with: Netherlands
The chosen file was already detected in the download directory. Skip downloading.
The corresponding gpkg file was already detected. Skip vectortranslate operations.
Reading query `SELECT * FROM lines WHERE osm_id IN (6429507,6439305,6439218,6439342,6437077,6437113,6437085,6437055,6437019,6437007)' from data source `/home/mtes/local/data/geofabrik_netherlands-latest.gpkg' using driver `GPKG'
Simple feature collection with 0 features and 9 fields
Bounding box: xmin: NA ymin: NA xmax: NA ymax: NA
Geodetic CRS: WGS 84
and
Simple feature collection with 0 features and 9 fields
Bounding box: xmin: NA ymin: NA xmax: NA ymax: NA
Geodetic CRS: WGS 84
[1] osm_id name highway waterway aerialway barrier man_made z_order other_tags geometry
<0 rows> (or 0-length row.names)
I downloaded oe_match("Netherlands").
Could you take a look @agila5 ? Thanks.
Hi @mtennekes. I'm sorry (and maybe I miss something obvious) but I cannot run your osmdata example:
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
dat1 <- opq_osm_id (type = "way", id = ids) %>%
opq_string () %>%
osmdata_sf ()
#> Error in opq_osm_id(type = "way", id = ids): id must be character or numeric.
Created on 2022-03-09 by the reprex package (v2.0.1)
Session info
sessioninfo::session_info()
#> - Session info ---------------------------------------------------------------
#> setting value
#> version R version 4.1.2 (2021-11-01)
#> os Windows 10 x64 (build 19042)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_United Kingdom.1252
#> ctype English_United Kingdom.1252
#> tz Europe/Berlin
#> date 2022-03-09
#> pandoc 2.16.2 @ C:/PROGRA~1/Pandoc/ (via rmarkdown)
#>
#> - Packages -------------------------------------------------------------------
#> package * version date (UTC) lib source
#> backports 1.4.1 2021-12-13 [1] CRAN (R 4.1.2)
#> cli 3.2.0 2022-02-14 [1] CRAN (R 4.1.2)
#> crayon 1.5.0 2022-02-14 [1] CRAN (R 4.1.2)
#> curl 4.3.2 2021-06-23 [1] CRAN (R 4.1.1)
#> digest 0.6.29 2021-12-01 [1] CRAN (R 4.1.2)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.1.1)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.1.1)
#> fansi 1.0.2 2022-01-14 [1] CRAN (R 4.1.2)
#> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.1.1)
#> fs 1.5.2 2021-12-08 [1] CRAN (R 4.1.2)
#> generics 0.1.2 2022-01-31 [1] CRAN (R 4.1.2)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.1.2)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.1.1)
#> htmltools 0.5.2 2021-08-25 [1] CRAN (R 4.1.1)
#> httr 1.4.2 2020-07-20 [1] CRAN (R 4.1.1)
#> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.1.2)
#> knitr 1.37 2021-12-16 [1] CRAN (R 4.1.2)
#> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.1.2)
#> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.1.1)
#> lubridate 1.8.0 2021-10-07 [1] CRAN (R 4.1.1)
#> magrittr 2.0.2 2022-01-26 [1] CRAN (R 4.1.2)
#> osmdata * 0.1.9.001 2022-03-09 [1] Github (ropensci/osmdata@bf0d639)
#> pillar 1.7.0 2022-02-01 [1] CRAN (R 4.1.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.1)
#> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.1.1)
#> R.cache 0.15.0 2021-04-30 [1] CRAN (R 4.1.1)
#> R.methodsS3 1.8.1 2020-08-26 [1] CRAN (R 4.1.1)
#> R.oo 1.24.0 2020-08-26 [1] CRAN (R 4.1.1)
#> R.utils 2.11.0 2021-09-26 [1] CRAN (R 4.1.1)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.1)
#> Rcpp 1.0.8 2022-01-13 [1] CRAN (R 4.1.2)
#> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.1.1)
#> rlang 1.0.2 2022-03-04 [1] CRAN (R 4.1.2)
#> rmarkdown 2.11 2021-09-14 [1] CRAN (R 4.1.1)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.1.1)
#> rvest 1.0.2 2021-10-16 [1] CRAN (R 4.1.1)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.1.2)
#> sp 1.4-6 2021-11-14 [1] CRAN (R 4.1.2)
#> stringi 1.7.6 2021-11-29 [1] CRAN (R 4.1.2)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.1.1)
#> styler 1.6.2 2021-09-23 [1] CRAN (R 4.1.1)
#> tibble 3.1.6 2021-11-07 [1] CRAN (R 4.1.2)
#> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.1.1)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.1.1)
#> withr 2.4.3 2021-11-30 [1] CRAN (R 4.1.2)
#> xfun 0.29 2021-12-14 [1] CRAN (R 4.1.2)
#> xml2 1.3.3 2021-11-30 [1] CRAN (R 4.1.2)
#> yaml 2.2.2 2022-01-25 [1] CRAN (R 4.1.2)
#>
#> [1] C:/Users/andre/Documents/R/win-library/4.1
#> [2] C:/Program Files/R/R-4.1.2/library
#>
#> ------------------------------------------------------------------------------
If I convert the input to numeric (i.e. non integer) or character I get a different error:
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
dat1 <- opq_osm_id (type = "way", id = as.numeric(ids)) %>%
opq_string () %>%
osmdata_sf ()
#> Error in opq_osm_id(type = "way", id = as.numeric(ids)): Only a single id may be entered.
dat1 <- opq_osm_id (type = "way", id = as.character(ids)) %>%
opq_string () %>%
osmdata_sf ()
#> Error in opq_osm_id(type = "way", id = as.character(ids)): Only a single id may be entered.
Created on 2022-03-09 by the reprex package (v2.0.1)
On the other hand, if I consider just a unique osm_id then everything works fine (vectortranslate_options is just a faster version of query argument in this case) :
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
library(osmextract)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright.
#> Check the package website, https://docs.ropensci.org/osmextract/, for more details.
ids = 6429507
dat1 <- opq_osm_id(id = ids, type = "way") %>%
opq_string () %>%
osmdata_sf ()
dat1
#> Object of class 'osmdata' with:
#> $bbox :
#> $overpass_call : The call submitted to the overpass API
#> $meta : metadata including timestamp and version numbers
#> $osm_points : 'sf' Simple Features Collection with 33 points
#> $osm_lines : 'sf' Simple Features Collection with 1 linestrings
#> $osm_polygons : 'sf' Simple Features Collection with 0 polygons
#> $osm_multilines : NULL
#> $osm_multipolygons : NULL
dat2 <- oe_get(
place = "Neatherlands",
vectortranslate_options = c("-where", "osm_id = '6429507'")
)
#> The input place was matched with: Netherlands
#> The chosen file was already detected in the download directory. Skip downloading.
#> Start with the vectortranslate operations on the input file!
#> 0...10...20...30...40...50...60...70...80...90...100 - done.
#> Finished the vectortranslate operations on the input file!
#> Reading layer `lines' from data source
#> `C:\Users\andre\Documents\osm-data\geofabrik_netherlands-latest.gpkg'
#> using driver `GPKG'
#> Simple feature collection with 1 feature and 9 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: 6.421445 ymin: 52.62376 xmax: 6.434561 ymax: 52.64008
#> Geodetic CRS: WGS 84
# Quick plot
par(mfrow = c(1, 2))
plot(dat1$osm_lines$geometry)
plot(dat2$geometry)
Created on 2022-03-09 by the reprex package (v2.0.1)
Do you have any idea what's going on? Can you paste the version of osmextract and osmdata packages?
Very strange. Also your dat2 example resulted in a 0-row sf object. Also strange that osmdata didn't work for you.
osmdata * 0.1.8 2021-10-17 [1] CRAN (R 4.1.2)
osmextract * 0.4.0 2021-10-27 [1] CRAN (R 4.1.2)
Which versions did you use? Any other things we can test?
You're right. I don't get an error with version 0.1.8
remotes::install_version("osmdata", "0.1.8", quiet = TRUE)
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
dat1 <- opq_osm_id (type = "way", id = ids) %>%
opq_string () %>%
osmdata_sf ()
dat1
#> Object of class 'osmdata' with:
#> $bbox :
#> $overpass_call : The call submitted to the overpass API
#> $meta : metadata including timestamp and version numbers
#> $osm_points : 'sf' Simple Features Collection with 57 points
#> $osm_lines : 'sf' Simple Features Collection with 10 linestrings
#> $osm_polygons : 'sf' Simple Features Collection with 0 polygons
#> $osm_multilines : NULL
#> $osm_multipolygons : NULL
Created on 2022-03-09 by the reprex package (v2.0.1)
but I get an error with the current CRAN (i.e. 0.1.9)
remotes::install_cran("osmdata", quiet = TRUE)
#> Installing 1 packages: osmdata
#> package 'osmdata' successfully unpacked and MD5 sums checked
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
dat1 <- opq_osm_id (type = "way", id = ids) %>%
opq_string () %>%
osmdata_sf ()
#> Error in opq_osm_id(type = "way", id = ids): id must be character or numeric.
dat1
#> Error in eval(expr, envir, enclos): object 'dat1' not found
Created on 2022-03-09 by the reprex package (v2.0.1)
and github version (i.e. 0.1.9.001):
remotes::install_github("ropensci/osmdata", quiet = TRUE)
library(osmdata)
#> Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright
ids = c(6429507L, 6439305L, 6439218L, 6439342L, 6437077L, 6437113L, 6437085L, 6437055L, 6437019L, 6437007L)
dat1 <- opq_osm_id (type = "way", id = ids) %>%
opq_string () %>%
osmdata_sf ()
#> Error in opq_osm_id(type = "way", id = ids): id must be character or numeric.
dat1
#> Error in eval(expr, envir, enclos): object 'dat1' not found
Created on 2022-03-09 by the reprex package (v2.0.1)
Tomorrow I will create a new issue in the osmdata repo. [Done, see: https://github.com/ropensci/osmdata/issues/268]
I still don't understand what's the problem with osmextract but I think it might be related to #245. Could you try installing the github version of osmextract, clean all files in the download directory and try again?
I've tried the GH version, but without success. Without knowledge of the file types, I noticed that gpkg is some kind of processed file. When I first start the query it returns the message "Start with the vectortranslate operations on the input file!" and takes a bit of time. When I rerun the query, it returns the (in my case empty) output immediately.
What I notices is that it gives warnings about being unable to open database:
> iow_subset <- oe_get(
+ place = "Netherlands",
+ query = paste0("SELECT * FROM lines WHERE osm_id IN (6429507)")
+ )
The input place was matched with: Netherlands
The chosen file was already detected in the download directory. Skip downloading.
Start with the vectortranslate operations on the input file!
0...10...20...30...40...50...60...70...80...90...100 - done.
Finished the vectortranslate operations on the input file!
Reading query `SELECT * FROM lines WHERE osm_id IN (6429507)' from data source `/home/mtes/local/data/geofabrik_netherlands-latest.gpkg' using driver `GPKG'
Simple feature collection with 0 features and 9 fields
Bounding box: xmin: NA ymin: NA xmax: NA ymax: NA
Geodetic CRS: WGS 84
There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 4: sqlite3_open(/home/mtes/local/data/geofabrik_netherlands-latest.gpkg) failed: unable to open database file
2: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 4: sqlite3_open(/home/mtes/local/data/geofabrik_netherlands-latest.gpkg) failed: unable to open database file
3: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 1: Cannot write in temporary node file ./osm_tmp_nodes_6185_3 : No such file or directory
4: In CPL_gdalvectortranslate(source, destination, options, ... :
And what happens when you run
osmextract::oe_get(
place = "Netherlands",
query = paste0("SELECT * FROM lines WHERE osm_id IN (6429507)"),
download_directory = tempdir()
)
in a fresh R session?
Doesn't work unfortunately:
> warnings()
Warning messages:
1: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 1: Cannot write in temporary node file ./osm_tmp_nodes_21125_1 : No such file or directory
2: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 1: Cannot write in temporary node file ./osm_tmp_nodes_21125_1 : No such file or directory
3: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 1: Cannot write in temporary node file ./osm_tmp_nodes_21125_1 : No such file or directory
4: In CPL_gdalvectortranslate(source, destination, options, ... :
GDAL Error 1: Cannot write in temporary node file ./osm_tmp_nodes_21125_1 : No such file or directory
5: In CPL_gdalvectortranslate(source, destination, options, ... :
Any other tests I can run?
What version of GDAL are you both running? I'm on 3.4.0
library(sf)
#> Linking to GEOS 3.10.1, GDAL 3.4.0, PROJ 8.2.0; sf_use_s2() is TRUE
Created on 2022-03-14 by the reprex package (v2.0.1)
Update, good news, it works for me :+1:
osmextract::oe_get(
+ place = "Netherlands",
+ query = paste0("SELECT * FROM lines WHERE osm_id IN (6429507)"),
+ download_directory = tempdir()
+ )
The input place was matched with: Netherlands
You are trying to download a file from https://download.geofabrik.de/europe/netherlands-latest.osm.pbf
This is a large file (1074 MB)!
Are you sure that you want to download it?
1: Yes
2: No
Selection: 1
|========================================================================================================================| 100%
File downloaded!
Start with the vectortranslate operations on the input file!
0...10...20...30...40...50...60...70...80...90...100 - done.
Finished the vectortranslate operations on the input file!
Reading query `SELECT * FROM lines WHERE osm_id IN (6429507)' from data source `/tmp/RtmpFWe8gC/geofabrik_netherlands-latest.gpkg' using driver `GPKG'
Simple feature collection with 1 feature and 9 fields
Geometry type: LINESTRING
Dimension: XY
Bounding box: xmin: 6.421445 ymin: 52.62376 xmax: 6.434561 ymax: 52.64008
Geodetic CRS: WGS 84
What version of GDAL are you both running? I'm on 3.4.0
I run 3.2.1
Any other tests I can run?
Can you run the examples in sf::gdal_utils() and report back? For example, what is the output of the following?
in_file <- system.file("shape/storms_xyz.shp", package="sf")
out_file <- paste0(tempfile(), ".gpkg")
gdal_utils(
util = "vectortranslate",
source = in_file,
destination = out_file, # output format must be specified for GDAL < 2.3
options = c("-f", "GPKG")
)
gdal_utils(
util = "vectortranslate",
source = in_file,
destination = out_file,
options = c(
"-f", "GPKG", # output file format for GDAL < 2.3
"-s_srs", "EPSG:4326", # input file SRS
"-t_srs", "EPSG:2264", # output file SRS
"-overwrite"
)
)
st_read(out_file)
st_read(in_file)
I'm using:
Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1; sf_use_s2() is TRUE
Now rebooting to Windows to test...
Yes, Windows works!
With:
Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1; sf_use_s2() is TRUE
I don't think so since the following
osmextract::oe_get(
place = "Isle of Wight",
query = "SELECT * FROM lines WHERE osm_id = '413'",
download_directory = tempdir()
)
#> Warning: st_crs<- : replacing crs does not reproject data; use st_transform for
#> that
#> The input place was matched with: Isle of Wight
#> | | | 0% | | | 1% | |= | 1% | |= | 2% | |== | 2% | |== | 3% | |== | 4% | |=== | 4% | |=== | 5% | |==== | 5% | |==== | 6% | |===== | 7% | |===== | 8% | |====== | 8% | |====== | 9% | |======= | 9% | |======= | 10% | |======= | 11% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 22% | |================ | 23% | |================= | 24% | |================= | 25% | |================== | 25% | |================== | 26% | |=================== | 26% | |=================== | 27% | |=================== | 28% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |===================== | 31% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================= | 34% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |=========================== | 39% | |============================ | 39% | |============================ | 40% | |============================ | 41% | |============================= | 41% | |============================= | 42% | |============================== | 42% | |============================== | 43% | |=============================== | 44% | |=============================== | 45% | |================================ | 45% | |================================ | 46% | |================================= | 46% | |================================= | 47% | |================================= | 48% | |================================== | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |=================================== | 51% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |===================================== | 54% | |====================================== | 54% | |====================================== | 55% | |======================================= | 55% | |======================================= | 56% | |======================================== | 57% | |======================================== | 58% | |========================================= | 58% | |========================================= | 59% | |========================================== | 59% | |========================================== | 60% | |========================================== | 61% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 62% | |============================================ | 63% | |============================================= | 64% | |============================================= | 65% | |============================================== | 65% | |============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 68% | |================================================ | 69% | |================================================= | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |================================================== | 72% | |=================================================== | 72% | |=================================================== | 73% | |==================================================== | 74% | |==================================================== | 75% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 76% | |====================================================== | 77% | |====================================================== | 78% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 79% | |======================================================== | 80% | |======================================================== | 81% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |========================================================== | 84% | |=========================================================== | 84% | |=========================================================== | 85% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 87% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 89% | |=============================================================== | 90% | |=============================================================== | 91% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 92% | |================================================================= | 93% | |================================================================== | 94% | |================================================================== | 95% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 97% | |==================================================================== | 98% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100%
#> File downloaded!
#> Start with the vectortranslate operations on the input file!
#> 0...10...20...30...40...50...60...70...80...90...100 - done.
#> Finished the vectortranslate operations on the input file!
#> Reading query `SELECT * FROM lines WHERE osm_id = '413'' from data source `/tmp/RtmpQWh3o3/geofabrik_isle-of-wight-latest.gpkg' using driver `GPKG'
#> Simple feature collection with 1 feature and 9 fields
#> Geometry type: LINESTRING
#> Dimension: XY
#> Bounding box: xmin: -1.083348 ymin: 50.68616 xmax: -1.082644 ymax: 50.68625
#> CRS: 4326
Created on 2022-03-15 by the reprex package (v2.0.1)
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 3.6.3 (2020-02-29)
#> os Ubuntu 18.04.6 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language en_GB:en
#> collate en_GB.UTF-8
#> ctype en_GB.UTF-8
#> tz Europe/Rome
#> date 2022-03-15
#> pandoc 2.17.1.1 @ /home/ag21052/.local/bin/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> assertthat 0.2.1 2019-03-21 [3] CRAN (R 3.5.3)
#> backports 1.4.1 2021-12-13 [3] CRAN (R 3.6.3)
#> class 7.3-20 2022-01-13 [4] CRAN (R 3.6.3)
#> classInt 0.4-3 2020-04-07 [3] CRAN (R 3.6.3)
#> cli 3.2.0 2022-02-14 [1] CRAN (R 3.6.3)
#> crayon 1.5.0 2022-02-14 [1] CRAN (R 3.6.3)
#> curl 4.3.2 2021-06-23 [3] CRAN (R 3.6.3)
#> DBI 1.1.2 2021-12-20 [3] CRAN (R 3.6.3)
#> digest 0.6.29 2021-12-01 [1] CRAN (R 3.6.3)
#> dplyr 1.0.8 2022-02-08 [1] CRAN (R 3.6.3)
#> e1071 1.7-9 2021-09-16 [1] CRAN (R 3.6.3)
#> ellipsis 0.3.2 2021-04-29 [3] CRAN (R 3.6.3)
#> evaluate 0.15 2022-02-18 [1] CRAN (R 3.6.3)
#> fansi 1.0.2 2022-01-14 [3] CRAN (R 3.6.3)
#> fastmap 1.1.0 2021-01-25 [1] CRAN (R 3.6.3)
#> fs 1.5.2 2021-12-08 [3] CRAN (R 3.6.3)
#> generics 0.1.2 2022-01-31 [1] CRAN (R 3.6.3)
#> glue 1.6.1 2022-01-22 [1] CRAN (R 3.6.3)
#> highr 0.9 2021-04-16 [3] CRAN (R 3.6.3)
#> htmltools 0.5.2 2021-08-25 [1] CRAN (R 3.6.3)
#> httr 1.4.2 2020-07-20 [3] CRAN (R 3.6.3)
#> KernSmooth 2.23-20 2021-05-03 [4] CRAN (R 3.6.3)
#> knitr 1.37 2021-12-16 [1] CRAN (R 3.6.3)
#> lifecycle 1.0.1 2021-09-24 [3] CRAN (R 3.6.3)
#> magrittr 2.0.2 2022-01-26 [1] CRAN (R 3.6.3)
#> osmextract 0.4.0.9000 2022-03-15 [1] local
#> pillar 1.7.0 2022-02-01 [1] CRAN (R 3.6.3)
#> pkgconfig 2.0.3 2019-09-22 [3] CRAN (R 3.6.1)
#> proxy 0.4-26 2021-06-07 [1] CRAN (R 3.6.3)
#> purrr 0.3.4 2020-04-17 [3] CRAN (R 3.6.3)
#> R.cache 0.15.0 2021-04-30 [1] CRAN (R 3.6.3)
#> R.methodsS3 1.8.1 2020-08-26 [3] CRAN (R 3.6.3)
#> R.oo 1.24.0 2020-08-26 [3] CRAN (R 3.6.3)
#> R.utils 2.11.0 2021-09-26 [3] CRAN (R 3.6.3)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 3.6.3)
#> Rcpp 1.0.8 2022-01-13 [3] CRAN (R 3.6.3)
#> reprex 2.0.1 2021-08-05 [3] CRAN (R 3.6.3)
#> rlang 1.0.1 2022-02-03 [1] CRAN (R 3.6.3)
#> rmarkdown 2.11 2021-09-14 [1] CRAN (R 3.6.3)
#> rstudioapi 0.13 2020-11-12 [3] CRAN (R 3.6.3)
#> sessioninfo 1.2.2 2021-12-06 [3] CRAN (R 3.6.3)
#> sf 1.0-5 2021-12-17 [3] CRAN (R 3.6.3)
#> stringi 1.7.6 2021-11-29 [3] CRAN (R 3.6.3)
#> stringr 1.4.0 2019-02-10 [3] CRAN (R 3.5.2)
#> styler 1.6.2 2021-09-23 [3] CRAN (R 3.6.3)
#> tibble 3.1.6 2021-11-07 [1] CRAN (R 3.6.3)
#> tidyselect 1.1.2 2022-02-21 [3] CRAN (R 3.6.3)
#> units 0.8-0 2022-02-05 [1] CRAN (R 3.6.3)
#> utf8 1.2.2 2021-07-24 [3] CRAN (R 3.6.3)
#> vctrs 0.3.8 2021-04-29 [3] CRAN (R 3.6.3)
#> withr 2.4.3 2021-11-30 [3] CRAN (R 3.6.3)
#> xfun 0.29 2021-12-14 [3] CRAN (R 3.6.3)
#> yaml 2.3.5 2022-02-21 [1] CRAN (R 3.6.3)
#>
#> [1] /home/ag21052/R/x86_64-pc-linux-gnu-library/3.6
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library
#>
#> ──────────────────────────────────────────────────────────────────────────────
works on my VM running
sf::sf_extSoftVersion()
#> GEOS GDAL proj.4 GDAL_with_GEOS USE_PROJ_H
#> "3.7.1" "2.4.2" "5.2.0" "true" "false"
#> PROJ
#> "5.2.0"
Created on 2022-03-15 by the reprex package (v2.0.1)
@mtennekes, if you want we could set a (skype/zoom/meet/whatever) call to check the problem together. I don't have any further suggestion unfortunately on why that doesn't work on your pc.
Thanks @agila5 I got the job done via Windows so there is no urgency anymore, but it would be nice to find out the issue. So a zoom would be nice. Then we can also share some research ideas. Also interested @Robinlovelace ?
Sure! My n. 1 priority would be upstream improvements to providers: it makes no sense that I can download West Yorkshire but not a region in India. Something @bdon can help with? Apologies for the out-of-the-blue tagging but Dustin Carlino suggested you may be able to help with something like this. I'm well up for a call in any case :+1:
@Robinlovelace you're welcome to link users to download PBF files via Web UI at http://protomaps.com/downloads - this is a free service that lets you extract any rectangular or polygonal region in the world.
The Web UI service however is rate-limited, and I provide programmatic access only for users in private and not to the general public. The reason for this is that the Protomaps Extracts service is generated 100% on demand, snapshotting a minute-level transaction, so can take a long time just to perform the indexing before streaming the PBF bytes out.
The other upstream providers have pre-specified areas that may be updated once a day or week, but since those are static files they're much less intensive to serve over the web.
My suggestion is: add docs and a first-class way for users to download a PBF "by hand", place it in the persistent directory and then reference it in the code. This would add some slight friction to the user experience but would let you effectively support any PBF source out there.
My suggestion is: add docs and a first-class way for users to download a PBF "by hand", place it in the persistent directory and then reference it in the code. This would add some slight friction to the user experience but would let you effectively support any PBF source out there.
Many thanks for the suggestion, this sounds good to me. Thoughts @agila5 ? I've not tried the workflow but it sounds promising.
Hi @bdon and thanks for your message.
Thoughts @agila5 ? I've not tried the workflow but it sounds promising.
I think that's a reasonable idea but I have one question. Am I right saying that, in general, it's not possible to download those PBF extracts directly from R? So the workflow should be something like 1) download the .osm.pbf file from the browser UI for a given rectangle/polygon; 2) read in the file using one of our functions; 3) manipulate/plot/whatever?
btw: @mtennekes, if you are still interested in a quick chat to chat about the aforementioned problem, I'm free on Tuesday/Wednesday/Thursday from 9AM to 6PM.
btw: @mtennekes, if you are still interested in a quick chat to chat about the aforementioned problem, I'm free on Tuesday/Wednesday/Thursday from 9AM to 6PM.
I'm still looking forward to meeting you @agila5, but have to postpone it to mid June (or later), when I will catch up with that project. Will let you know! Btw, I also recently switched from laptops (in my case from Ubuntu/Windows dual boot to Mac). The problem I had was only on Ubuntu, so cannot reproduce it anymore.
I'm still looking forward to meeting you @agila5, but have to postpone it to mid June (or later), when I will catch up with that project.
Ok, no problem! Send me a message when you start working again with OSM data.
so cannot reproduce it anymore.
Ops ahahaha No worries then, we will just wait if there is any new issue from Ubuntu users.