openaq-fetch icon indicating copy to clipboard operation
openaq-fetch copied to clipboard

New South Wales, Australia incorrect history

Open andrewharvey opened this issue 5 years ago • 2 comments

continuing from #646, the NSW, AU data was previously scrapped incorrectly so the data is incorrect, this needs fixing. Quoting from slack https://openaq.slack.com/archives/C098H3JH2/p1576814663003400

...Removing the data and try to do an invest of the older data may be a good way to go, with one caveat. Our system currently has no idea of whether the data is qa/qc’d, but only that it’s real-time. If the older data is not the same ‘class’ as what’s available in real-time, we’d be mixing two different classes of data which may be an issue. Andrew Harvey 1 hour ago ...I'll try to compare the historical data to the live to see if there's a difference.

This is different to #647, which is about loading historical data from even before we started ingesting invalid data.

andrewharvey avatar Dec 20 '19 23:12 andrewharvey

@russbiggs I am not sure if we can fix the data if it was scraped incorrectly (adapter: "Australia - New South Wales" nsw.js) - unless the source provides historical data. Currently I cannot get the website to resolve https://airquality.environment.nsw.gov.au/ . We should consider erasing the data from this source prior to Dec 19 2019, if not just the entries for so2 from issue #646 (source name: State of New South Wales (Department of Planning, Industry and Environment)

majesticio avatar Feb 02 '23 17:02 majesticio

https://airquality.environment.nsw.gov.au/aquisnetnswphp/getPage.php?reportid=2 is working for me

andrewharvey avatar Feb 03 '23 03:02 andrewharvey