GTFS icon indicating copy to clipboard operation
GTFS copied to clipboard

In Routes.txt the route_desc should be optional but it appears manditory.

Open rhwilburn opened this issue 9 years ago • 5 comments

    protected virtual Route ParseRoute(T feed, GTFSSourceFileHeader header, string[] data)
    {
        // check required fields.
        this.CheckRequiredField(header, header.Name, this.RouteMap, "route_id");

        this.CheckRequiredField(header, header.Name, this.RouteMap, "route_short_name");
        this.CheckRequiredField(header, header.Name, this.RouteMap, "route_long_name");
        this.CheckRequiredField(header, header.Name, this.RouteMap, "route_desc"); //<------ This one
        this.CheckRequiredField(header, header.Name, this.RouteMap, "route_type");

See here: https://developers.google.com/transit/gtfs/reference?hl=en#routes_fields

rhwilburn avatar Jan 13 '16 01:01 rhwilburn

I have a related scenario where I have line like thus:

image

Where you can see the data array has 7 elements. The header however has 8 elements as it includes route_desc which causes issues (only for the one row with double quotes around the first cell. Most of my rows work fine interestingly appart from row 236 which only seems odd as it has quotes in the first field as thus (it has quotes because it has a comma in the name which is what is causing issues I think):

"HC Station to Orewa, via Silverdale and Red Beach Rd" ,3 , ,NZBNS ,98402-20151209115617_v35.34 , ,984

All other rows are more like:

Carmel College And Westlake Girls To Glenfield ,3 , ,NZBNS ,04652-20151126154057_v35.16 , ,046

The headers in the file are:

route_long_name ,route_type,route_text_color,agency_id ,route_id,route_color,route_short_name

I think the issues are:

  1. The manditory check doesn't seem to enforce that column is there. It creates that column if its not. It probably shouldn't create the column if its not manditory.
  2. How can a comma be supported (can the double quotes functionality be implemented so that it is tokenizing on commas not inside quotes? I think the double quotes should be removed too once the commas are within data cells as the quotes are only intended for use to denote the extent of a token.

rhwilburn avatar Jan 13 '16 01:01 rhwilburn

Can you create a unittest that reproduces these issues? It's difficult to follow without access to the data you're using.

xivk avatar Jan 13 '16 12:01 xivk

Firstly issue number 1, If you change your routes.txt in the unit test project to be (ie remove the route_desc column):

route_id,agency_id,route_short_name,route_long_name,route_type,route_url,route_color,route_text_color AB,DTA,10,Airport - Bullfrog,,3,C4008F, BFC,DTA,20,Bullfrog - Furnace Creek Resort,3,,#FFFFFF, STBA,DTA,30,Stagecoach - Airport Shuttle,3,,, CITY,DTA,40,City,3,,, AAMV,DTA,50,Airport - Amargosa Valley,3,,,

Then it will give the following error:

GTFS.Exceptions.GTFSRequiredFieldMissingException : Required field route_desc not found in routes.

at GTFS.GTFSReader1.CheckRequiredField(GTFSSourceFileHeader header, String name, FieldMap fieldMap, String column) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 1286 at GTFS.GTFSReader1.ParseRoute(T feed, GTFSSourceFileHeader header, String[] data) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 878 at GTFS.GTFSReader1.Read[TEntity](IGTFSSourceFile file, T feed, EntityParseDelegate1 parser, EntityAddDelegate1 addDelegate) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 466 at GTFS.GTFSReader1.Read(IGTFSSourceFile file, T feed) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 390 at GTFS.GTFSReader1.Read(T feed, IEnumerable1 source, IGTFSSourceFile file) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 284 at GTFS.GTFSReaderExtensions.Read[T](GTFSReader1 reader, IEnumerable1 source, IGTFSSourceFile file) in C:\Users\Richard\Desktop\GTFS-master\GTFS\GTFSReader.cs:line 1756 at GTFS.Test.DirectorySourceTest.ParseRoutes() in C:\Users\Richard\Desktop\GTFS-master\GTFS.Test\DirectorySourceTest.cs:line 91

That field should be optional according to the GTFS spec.

rhwilburn avatar Jan 13 '16 20:01 rhwilburn

For issue number 2 I can no longer reproduce this. The quotations do correctly tokenize the comma within (at least when trying through unit testing). I will keep an eye out for the issue and how to repro if I can. For now issue number 2 mentioned above can be ignored. Thanks.

rhwilburn avatar Jan 13 '16 20:01 rhwilburn

I have fixed the first issue, see issue #30.

rorlic avatar Feb 10 '16 11:02 rorlic