data-infra
data-infra copied to clipboard
cal-itp
Cal-ITP data infrastructure
# Description _Describe your changes and why you're making them. Please include the context, motivation, and relevant dependencies._ We've cycled through several offboarding of contractors and consultants and we should...
# Description Added a new sample notebook to show how to make a parameterized notebooks. Updated the docs with the link to this new notebook, as well as some additional...
The following modifications are suggested: - [x] add agency_id from routes.txt or ageny_name from agencies.txt - [ ] add a description for earliest_tap: `The earliest transaction associated with a customer_id.`...
In #2773, we identify two buckets that we want to move to cold storage (`gtfs-data` and `gtfs-data-test`). Since the ticket's creation, we've determined that applying GCP's [AutoClass](https://cloud.google.com/storage/docs/autoclass) feature to these...
## User story / feature request In order to comply with Caltrans security parameters, we should remove all sensitive Elavon data from our data pipeline. 1. We need to remove...
## User story / feature request In order to comply with Caltrans security parameters, we should remove all sensitive Littlepay data from our data pipeline. 1. We need to remove...
As a Cal-ITP data user, I want obsolete Payments data models to be deprecated from the data warehouse so it's clear which data is current and maintained for active use....
After Littlepay's recent adjustment to their publishing cadence to better suit our analytics needs, we found that the new publishing time was too late for our `transform_warehouse` DAG start time...
Following rollout of V2 open data publishing, there remain a few things to take care of in our underlying tables and associated code when we get the chance. These include:...
As an analytics engineer, I want all of our columns to be documented in dbt so that future maintainers and users of the warehouse will understand what each column is...