data-infra icon indicating copy to clipboard operation
data-infra copied to clipboard

GTFS Data: Should we add URIs for GTFS for "future" service

Open e-lo opened this issue 3 years ago • 4 comments

? Should we update agencies.yml in order to also validate GTFS for service changes happening in the future?

If so, we should discuss how/where to add the following gtfs datasets in the agencies.yml data model.

  • https://gitlab.com/LACMTA/gtfs_bus/-/blob/future-service/gtfs_bus.zip
  • https://gitlab.com/LACMTA/gtfs_rail/-/blob/future-service/gtfs_rail.zip

For now, I have added the property future URI to the airtable California Transit.gtfs datasets

Checklist

  • [ ] Manually made sure any new feeds have itp_ids that are not duplicative
  • [ ] Mannually confirmed URIs are valid (to be automated in future per #908)
  • [ ] Made sure the Airtable database has consistent information

e-lo avatar Feb 12 '22 00:02 e-lo

Should we update agencies.yml in order to also validate GTFS for service changes happening in the future?

My personal feeling is that this is fairly low on the priority list - but something to track.

e-lo avatar Feb 12 '22 00:02 e-lo

Do these exist in airtable? I don't think it'd be much effort to add to agencies.yml and everything from there should sort of just work I think.

evansiroky avatar Feb 12 '22 01:02 evansiroky

I added these to the field 'future URI'

On Fri, Feb 11, 2022, 5:31 PM Evan Siroky @.***> wrote:

Do these exist in airtable? I don't think it'd be much effort to add to agencies.yml and everything from there should sort of just work I think.

— Reply to this email directly, view it on GitHub https://github.com/cal-itp/data-infra/issues/1075#issuecomment-1036917965, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGCBI4NMKSUAMMESJHAWPDU2WZ6JANCNFSM5OF24QBQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

e-lo avatar Feb 12 '22 03:02 e-lo

In the v2 pipeline, to download future data we could either:

  • Create separate datasets for future service and check them as yes for data pipeline
  • Rework the downloader to look at the future uri field and download it too Either case would likely also require moderately substantial modeling in the warehouse to suppress data from future datasets from the core GTFS tables.

lauriemerrell avatar Aug 28 '23 18:08 lauriemerrell