data-infra icon indicating copy to clipboard operation
data-infra copied to clipboard

Remove sensitive Littlepay data from Pipeline

Open evansiroky opened this issue 9 months ago • 3 comments

User story / feature request

In order to comply with Caltrans security parameters, we should remove all sensitive Littlepay data from our data pipeline.

  1. We need to remove data we have already collected from raw data and any BigQuery tables that ingested the data.
  2. We need to modify the DAG task that ingests Littlepay Data so that it does store any sensitive data in our cloud system.

Acceptance Criteria

The sensitive data that needs to be removed includes:

  1. customer_id? This one is a little bit of a gray area since it is a hash and directly personally identifiable.
  2. masked_pan

Notes

This issue can be separated into 2 phases:

  1. Scope the needed changes to implement the necessary changes and share with Caltrans IT Security for review.
  2. Implement the changes.

evansiroky avatar Apr 26 '24 18:04 evansiroky