data.gov
data.gov copied to clipboard
Create harvest runner main function for dcatus
User Story
In order for the harvest runner to process a harvest source, datagov wants to create a main
function in harvest.py
responsible for processing a harvest source.
Acceptance Criteria
[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]
- [ ] GIVEN harvest.py
WHENmain
is called
THEN a dcatus source should be harvested
Background
[Any helpful contextual notes or links to artifacts/evidence, if needed]
Security Considerations (required)
[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]
Sketch
- main function steps
- parse sys.argv and get the jobid
- pull the associated harvest source information from the DB via the jobid
- create a
HarvestSource
instance using the derived harvest source information - get records changes
- synchronize records
- get a simple harvest source as a test
- this'll be an integration test [Notes or a checklist reflecting our understanding of the selected approach]