data.gov icon indicating copy to clipboard operation
data.gov copied to clipboard

Create harvest runner main function for dcatus

Open rshewitt opened this issue 10 months ago • 0 comments

User Story

In order for the harvest runner to process a harvest source, datagov wants to create a main function in harvest.py responsible for processing a harvest source.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

  • [ ] GIVEN harvest.py
    WHEN main is called
    THEN a dcatus source should be harvested

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

  • main function steps
    • parse sys.argv and get the jobid
    • pull the associated harvest source information from the DB via the jobid
    • create a HarvestSource instance using the derived harvest source information
    • get records changes
    • synchronize records
  • get a simple harvest source as a test
  • this'll be an integration test [Notes or a checklist reflecting our understanding of the selected approach]

rshewitt avatar Apr 24 '24 16:04 rshewitt