the_od_bods icon indicating copy to clipboard operation
the_od_bods copied to clipboard

Replace .csv in dcat api

Open KarenJewell opened this issue 2 years ago • 2 comments

Is your feature request related to a problem? Please describe. Currently, the dcat api script (dcat.py) outputs to a .csv which is then ingested into merge_data.py. This should be converted to a .json output to keep consistent with the rest of the project.

Describe the solution you'd like

  • [ ] Change the output of dcat.py to be a .json file
  • [ ] Update merge_data.py to ingest the new .json file
  • [ ] format code with Black before PR
  • [ ] update docs (architecture.md and datapipeline.png)

Describe alternatives you've considered None

Additional context Original ticket triggering this change is #163

KarenJewell avatar Feb 18 '23 15:02 KarenJewell

please assign the issue.. would love to work on it!

thiyagarajan17 avatar Feb 18 '23 15:02 thiyagarajan17

@thiyagarajan17 I did some work on this to do some bug fixes and had forgotten this issue was assigned to you. I've made a start on it if you've got any comments/feedback?

https://github.com/OpenDataScotland/the_od_bods/tree/jack-dcat-improvements

JackGilmore avatar Apr 30 '23 15:04 JackGilmore