reach icon indicating copy to clipboard operation
reach copied to clipboard

Dev: Modifications to make local dev easier

Open jdu opened this issue 5 years ago • 1 comments

The following changes to the Reach airflow DAG in order to make local development / investigations / debugging easier.

  1. [ ] If Reach DEBUG=1 env is set, save DAG tasks locally to a configured folder instead of S3 (this is to avoid local runs overwriting each other in S3 between developers)
  2. [ ] Check for local EPMC dataset file and load EPMC from there.
  3. [ ] Pull EPMC records for policy-mock to reduce load for EPMC task in the DAG and speed up testing.
  4. [ ] Hide policy-mock in staging / production as the test server isn't running in staging/prod
  5. [ ] Add more documents to the Acme test scrape target for easier local testing

jdu avatar Jan 17 '20 10:01 jdu

This may be partially addressed by architectural changes proposed in #419

jdu avatar Feb 13 '20 08:02 jdu