ERROR c.c.g.GTFSFeed - MapDB database was not closed before it was garbage collected. This is a bug!

[temospena]

Hi,

I'm trying to use a gtfs file with the setup_r5r. When I run the setup, with the gtfs.zip file and a raster.tif file in the folder, it gives me an error:

r5r_lts_elevation = setup_r5(data_path = "r5r/", elevation = "MINETTI", overwrite = TRUE)

Using cached R5 version from C:/Users/Utilizador/AppData/Local/R/win-library/4.2/r5r/jar/r5-v6.7-all.jar 2022-06-29 20:05:00,051 [main] ERROR c.c.r.s.StreetLayer - Continuing to load but ignoring generalized costs due to exception: java.lang.RuntimeException: All ways are expected to have generalized cost tags. Missing: slope_1:forward jun 29, 2022 8:05:07 DA TARDE org.hsqldb.persist.Logger logInfoEvent INFO: dataFileCache open start Error in setup_r5(data_path = "r5r/", elevation = "MINETTI", overwrite = TRUE) : com.conveyal.gtfs.GtfsLibException: Error loading GTFS.

The gtfs is from an official data source, and I already previously validated it, following #284 . There are some warnings present, but no errors.

When trying to run the same command again, the error message is another one:

Using cached R5 version from C:/Users/Utilizador/AppData/Local/R/win-library/4.2/r5r/jar/r5-v6.7-all.jar 2022-06-29 20:09:07,317 [Finalizer] ERROR c.c.g.GTFSFeed - MapDB database was not closed before it was garbage collected. This is a bug! WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.mapdb.Volume$ByteBufferVol (file:/C:/Users/Utilizador/AppData/Local/R/win-library/4.2/r5r/jar/r5-v6.7-all.jar) to method java.nio.DirectByteBuffer.cleaner() WARNING: Please consider reporting this to the maintainers of org.mapdb.Volume$ByteBufferVol WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Error in setup_r5(data_path = "r5r/", elevation = "MINETTI", overwrite = TRUE) : java.io.IOError: java.io.IOException: Wrong index checksum, store was not closed properly and could be corrupted.

If no gtfs.zip is used, the process runs smoothly.

I am using r5r 0.7.9000.

Any clue on this? Thank you!

[rafapereirabr]

Hi Rosa, tahnk you for opoening this issue. Does it work with a different GTFS?

[mvpsaraiva]

Hi @temospena. I did some tests here and the problem is indeed the GTFS, despite the validator saying it's ok. I've unzipped the GTFS, deleted the frequencies.txt (which was empty) and the shapes.txt (which appeared to be broken), then zipped it back into a new GTFS. Then, R5 was able to load the new GTFS with no errors.

[temospena]

Thank you Marcus!

Just to confirm, I will need the shapes.txt anyway to get it as part of the network, right? I will try to find an old version of the same provider, and let you know if it is working.

[mvpsaraiva]

The shapes.txt is optional, and route shapes are only used in the detailed_itineraries() function to compute travel distances and trip geometries. If you're using any of the other of r5r's functions, shapes are ignored.

[temospena]

I am in fact using detailed_iteneraries(), but I used an older version of the same gtfs.zip and it worked nicely for this purpose 🙂

Just a side note: I made some previous tests, and MINETTI elevation option is better for cycling in this area. But it took about 24h to run the setup_r5() process, with that gtfs.zip and with elevation (option 'MINETTI'). It is not a very large osm area (Lisbon region), but this process with 'MINETTI' takes maybe 20 or 30x more time than the use of 'TOBLER' , and it is constantly throwing this messages:

Gradient over 0.5: -0,690000 (dx 10,000000 / dy -6,900000) Clamping.
Gradient over 0.5: -0,740284 (dx 9,726000 / dy -7,200000) Clamping.
Gradient over 0.5: 0,690000 (dx 10,000000 / dy 6,900000) Clamping.
Gradient over 0.5: 0,740284 (dx 9,726000 / dy 7,200000) Clamping.
Gradient over 0.5: -0,610000 (dx 10,000000 / dy -6,100000) Clamping.
Gradient over 0.5: 0,903452 (dx 7,416000 / dy 6,700000) Clamping.
Gradient over 0.5: 0,610000 (dx 10,000000 / dy 6,100000) Clamping.
Gradient over 0.5: -0,903452 (dx 7,416000 / dy -6,700000) Clamping.
Gradient over 0.5: -1,200000 (dx 10,000000 / dy -12,000000) Clamping.
Gradient over 0.5: 1,200000 (dx 10,000000 / dy 12,000000) Clamping.
Gradient over 0.5: 0,530000 (dx 10,000000 / dy 5,300000) Clamping.
Gradient over 0.5: -0,720000 (dx 10,000000 / dy -7,200000) Clamping.
Gradient over 0.5: 0,630000 (dx 10,000000 / dy 6,300000) Clamping.
Gradient over 0.5: -0,530000 (dx 10,000000 / dy -5,300000) Clamping.
Gradient over 0.5: 0,720000 (dx 10,000000 / dy 7,200000) Clamping.
Gradient over 0.5: -0,630000 (dx 10,000000 / dy -6,300000) Clamping.
Gradient over 0.5: 0,641752 (dx 7,947000 / dy 5,100000) Clamping.
Gradient over 0.5: -0,641752 (dx 7,947000 / dy -5,100000) Clamping.
Gradient over 0.5: 0,500000 (dx 10,000000 / dy 5,000000) Clamping.

I was using options(java.parameters = '-Xmx16G'), but not sure if the time decreases that much if using 32GB or more. I can try later.

[mvpsaraiva]

We noticed that the Minetti function throws those messages, but we are unable to silence them from r5r. But we hadn't noticed that it takes that much longer to compute than Tobler. I'll take a look and check if there's something we can do. What elevation raster are you using?

[temospena]

I am using Copernicus, clipped for that area an re-projected to have the same crs than OSM.

[rafapereirabr]

This has been fixed in the dev version