data-infra
data-infra copied to clipboard
Remove sensitive Littlepay data from Pipeline
User story / feature request
In order to comply with Caltrans security parameters, we should remove all sensitive Littlepay data from our data pipeline.
- We need to remove data we have already collected from raw data and any BigQuery tables that ingested the data.
- We need to modify the DAG task that ingests Littlepay Data so that it does store any sensitive data in our cloud system.
Acceptance Criteria
The sensitive data that needs to be removed includes:
-
customer_id
? This one is a little bit of a gray area since it is a hash and directly personally identifiable. -
masked_pan
Notes
This issue can be separated into 2 phases:
- Scope the needed changes to implement the necessary changes and share with Caltrans IT Security for review.
- Implement the changes.