Tyler Burton

Routes and order subject to change, but it's defined in Steps 23 & 24 here https://raw.githubusercontent.com/GSA/datagov-harvesting-logic/main/docs/diagrams/mermaid/dest/etl_pipeline-1.svg

> #this is where I have pause and things start to breakdown in a way where we're starting to call the things apscheduler is supposed to handle If I'm following...

IMO the Job Table needs to exit as persistent storage for historical metrics. Given the above questions, it feels like a good time to ask exactly what we're planning to...

we can park this work for the time being since we're not moving forward with apscheduler.

This is for going forward only

Todo: resolve the question of Poetry compatibility with Cloud.gov

this might be superceded by a revised design which init's tasks per job

kicking this back to the board so we can focus on tickets that achieve our MVP-load-test

CKAN has a bulk delete. This could prove helpful: https://github.com/ckan/ckanapi#bulk-delete

What do we need to do to get this merged @robert-bryson?