Tyler Burton

Harden WAF ETL pipeline

4

## User Story In order to harvest WAF sources effectively and at scale, datagovteam would like to harden the current WAF ETL pipeline. ## Acceptance Criteria [ACs should be clearly...

Replace Broken Link Checker

16

## User Story In order to ensure the quality of our sites, datagovteam would like a reliable report on broken links. ## Acceptance Criteria - [ ] GIVEN a list...

Refresh UI header and footer in catalog.data.gov to mirror data.gov changes

1

## User Story In order to present a unified UI appearance, datagovteam wants to update the catalog UI, specifically the header and footer, to mirror the recent changes to data.gov....

Implement Error Handling in Datagov-Harvesting-Logic

19

## User Story In order to effectively communicate harvesting errors back to our agency partners, datagovteam wants to implement enhanced error handling in datagov-harvesting-logic. ## Acceptance Criteria [ACs should be...

Create ISO reader for MDTranslator

10

## User Story In order to transform an ISO-19115 source to DCAT-US, datagovteam needs to work with the MDTranslator team to implement an ISO-19115 reader for the [MDTranslator](https://github.com/adiwg/mdTranslator). ## Acceptance...

Create Basic Flask Harvest Administration App

10

## User Story In order to add new harvest sources to our Harvest DB, datagovteam wants to create a new Flask app to handle basic CRUD of Harvest Source configs,...

Transform FGDC/CSDGM source to DCAT-US

3

## User Story In order to test another facet of the transformation step in our pipeline, datagovteam would like to transform an FGDC/CSDGM WAF source to DCAT-US and push that...

Refactor Airflow ETL Pipeline DAG to incorporate feedback changes

5

## User Story In order to incorporate updates to the datagov-harvesting-logic API, and feedback from the most recent design sessions, changes need to be made to the Airflow ETL pipeline...

## User Story In order to manage large numbers of harvest jobs, data.gov wants to define a series of queue systems using Redis. ## Acceptance Criteria First we will define...

Configure S3 to store Harvester Records

2

## User Story In order to perform operations on the data records reliably, datagov wants an interface to interact with S3 or the localstack equivalent. ## Acceptance Criteria [ACs should...