gtfs_manager icon indicating copy to clipboard operation
gtfs_manager copied to clipboard

Provide additonal examples of sample data

Open maxwell8888 opened this issue 3 years ago • 9 comments

I will try to get the app to a point that it should work with any spec compliant data, however it is probably also worth adding some links to additional data sources that have been directly tested. This makes it easier for people who may not otherwise know where to source GTFS data, and of course helps ensure the data actually loads correctly.

Todo: Errors opening bus data from https://www.bus-data.dft.gov.uk/ Possibly find a London Underground GTFS as many people are familiar with it and it has colored routes.

maxwell8888 avatar Aug 23 '22 11:08 maxwell8888

I don't know if it can help but I also noticed problems opening data from the entire Paris region (https://transport.data.gouv.fr/datasets/horaires-prevues-sur-les-lignes-de-transport-en-commun-dile-de-france-gtfs-datahub/#dataset-resources).

image

woshilapin avatar Aug 29 '22 14:08 woshilapin

This is because the app did not handle the case where shapes.txt is missing. I've just pushed a commit which adds a fallback to create trip paths from stop coordinates instead.

The second problem is that the Paris dataset is too large for the app. At the moment the app will struggle with datasets larger than 20mb. I tried to open the Paris dataset and crashed my laptop, but you are welcome to have a go yourself. Unfortunately improving the App to handle datasets larger datasets isn't something I have planned in the near future.

maxwell8888 avatar Aug 29 '22 16:08 maxwell8888

I've just pushed a commit which adds a fallback to create trip paths from stop coordinates instead.

Impressive work-around, many thanks for keeping this going, keen to see how it handles other GTFS datasets.

One question comes to mind: Are there any tools for breaking-up GTFS datasets out-of-memory, e.g. based on geographic region or subnetworks?

Robinlovelace avatar Aug 29 '22 16:08 Robinlovelace

Well, it works on my machine, thanks for the reactivity image

woshilapin avatar Aug 29 '22 16:08 woshilapin

Beautiful haha! Do you mind sharing what operating system you are using?

maxwell8888 avatar Aug 29 '22 16:08 maxwell8888

I've just pushed a commit which adds a fallback to create trip paths from stop coordinates instead.

Impressive work-around, many thanks for keeping this going, keen to see how it handles other GTFS datasets.

One question comes to mind: Are there any tools for breaking-up GTFS datasets out-of-memory, e.g. based on geographic region or subnetworks?

Not that I'm aware of but it would be pretty easy to code this up, the hard part would be providing a UI to allow users to select a region. Ofcourse you could just provide a separate CLI tool or something but that's not great for non technical users. At the end of the day it shouldn't be necessary since there is no fundamental reason the app shouldn't be able to handle these kinds of datasets.

maxwell8888 avatar Aug 29 '22 16:08 maxwell8888

Not that I'm aware of but it would be pretty easy to code this up, the hard part would be providing a UI to allow users to select a region. Ofcourse you could just provide a separate CLI tool or something but that's not great for non technical users. At the end of the day it shouldn't be necessary since there is no fundamental reason the app shouldn't be able to handle these kinds of datasets.

I guess that's functionality that gtfs_manager could one day provide!

Robinlovelace avatar Aug 29 '22 16:08 Robinlovelace

Beautiful haha! Do you mind sharing what operating system you are using?

Sure. I’m on Ubuntu 22.04.

woshilapin avatar Aug 29 '22 17:08 woshilapin

One question comes to mind: Are there any tools for breaking-up GTFS datasets out-of-memory, e.g. based on geographic region or subnetworks?

Not that I'm aware of but it would be pretty easy to code this up, the hard part would be providing a UI to allow users to select a region. Ofcourse you could just provide a separate CLI tool or something but that's not great for non technical users. At the end of the day it shouldn't be necessary since there is no fundamental reason the app shouldn't be able to handle these kinds of datasets.

It's not that straightforward, as such a tool will have to keep up with the ever-increasing complexity of GTFS and it's extensions, e.g. {levels,pathways,transfers}.txt, GTFS-Fares v2, GTFS-Flex.

OneBusAway's transformer CLI can be used, e.g. as follows:

{"op":"retain", "match":{"file": "stops.txt", "stop_lat": "r/50.8/54.05/", "stop_lon": "r/10.7/15.2/"}}

derhuerst avatar Aug 29 '22 22:08 derhuerst