reach
reach copied to clipboard
Dev: Modifications to make local dev easier
The following changes to the Reach airflow DAG in order to make local development / investigations / debugging easier.
- [ ] If Reach DEBUG=1 env is set, save DAG tasks locally to a configured folder instead of S3 (this is to avoid local runs overwriting each other in S3 between developers)
- [ ] Check for local EPMC dataset file and load EPMC from there.
- [ ] Pull EPMC records for policy-mock to reduce load for EPMC task in the DAG and speed up testing.
- [ ] Hide policy-mock in staging / production as the test server isn't running in staging/prod
- [ ] Add more documents to the Acme test scrape target for easier local testing
This may be partially addressed by architectural changes proposed in #419