openaq-fetch icon indicating copy to clipboard operation
openaq-fetch copied to clipboard

UK DEFRA Data - seeking contact suggestions for SOS API

Open RocketD0g opened this issue 7 years ago • 1 comments

I'm seeking suggestions on a contact to confirm whether DEFRA's SOS API has a way to give latest measurements smoothly.

This convo started in a previous (now closed) issue when our scraper broke: https://github.com/openaq/openaq-fetch/issues/292

Below is from @jflasher, quoted from that thread:


Ok, DEFRA source (via scraping) should now be fixed. Did a quick look at the other two methods mentioned above as it'd be nice to use some other than scraping. A few quick observations.

Atom Feed

  • It looks like we'd have to follow a path like https://uk-air.defra.gov.uk/data/atom-dls/auto/2017/atom.en.xml -> https://uk-air.defra.gov.uk/data/atom-dls/auto/2017/GB_FixedObservations_2017_ABD.atom.en.xml -> https://uk-air.defra.gov.uk/data/atom-dls/observations/auto/GB_FixedObservations_2017_ABD.xml.
  • There are coordinates in here, which is good (well they're a bounding box, but we can make a center point from that)
  • We'd have to do this for each of the ~160 locations, that seems way too heavy to do on each fetch
  • Each of the final xmls provides all data since the beginning of the year, again way to heavy to do every 10 minutes.

Unless I'm missing something, doesn't look like this would be a good alternative.

SOS API

This looks really promising, it should allows us to get the data via a RESTful API and consume JSON 💃 . The base url looks like https://uk-air.defra.gov.uk/sos-ukair/api/v1/. Did a quick check of stations and it reported back a whole bunch of things, which is great. There are coordinates. I'm unsure of how best to achieve our use case though. We don't want to have to query the system 500 times to get 3 pollutants from ~160 stations. Would be good to try and reach out to someone over there and see how to best handle that.

RocketD0g avatar Apr 25 '17 18:04 RocketD0g

I believe we may be able to use the api to enhance our DEFRA adapter using the docs. Since we already have an adapter this is low priority

majesticio avatar Jan 26 '23 22:01 majesticio