Joseph LeGrand

Move from local download and PostgreSQL COPY command to a cloud-based Extract/Load workflow

1

# Problem Statement - [ ] Set up a Google Cloud Storage (GCS) bucket. - [ ] Likely need to configure BigQuery access to GCS bucket - and document how...

502 error when trying to download FDA UNII data

4

# Problem Statement We already have a view that uses DailyMed SPL data via RxNorm data files to go from NDC9 -> inactive ingredients (excipients): `int_mthspl_products_to_inactive_ingredients`. It is created via...

# Problem Statement We currently export mart flatfiles to AWS s3 buckets thanks to the diligent work of our interns Miles and Komal. With the move to BigQuery, it makes...

# Problem Statement We no longer need pgAdmin since we will be using BigQuery. Airflow probably will still use Postgres locally - it automatically spins up a Postgres container or...

# Problem Statement Moving from PostgreSQL to BigQuery means we will definitely find SQL that throws errors d/t the different systems. We need to identify these areas and modify the...

Weekly data quality checks for mart updates

3

Depends on #262 and to a lesser extent #266 # Problem Statement Without manual analysis, it's hard to know what changed week to week in our data marts. # Criteria...

# Problem Statement We need a wholesale review of our current Airflow DAGs and dbt setup for each data source from beginning to end. # Criteria for Success Every checkbox...

Full Dailymed DAG runs into error

1

# Problem Statement When trying to run DailyMed Full DAG (50,000 SPLs), I run into the error shown below. This is from the build in branch jrlegrand/dailymed # Criteria for...

PIP additional requirements throws Docker error

1

# Problem Statement Right now, we have CMS Part D DAG in a "hidden_dags" folder because it was causing an error for me and at least one other person when...

# Problem Statement We arbitrarily chose the first day of the month for the RxNorm DAG schedule, but it looks like it can be updated on days 1-6 or so...