openaq-fetch
openaq-fetch copied to clipboard
openaq
Korea AQ data
I found this and haven't yet had a chance to investigate closely: http://www.airkorea.or.kr/index
And it looks like PM10 and PM2.5 are shared in physical concentrations on map that may or may not be easy to determine coordinates.
Can anyone translate Korean?
I looked into Korea a few months ago and I had a hard time finding an API. They have an english version of this map though. Let me see if I can find it.
ah, perhaps it was just the seoul data. http://english.seoul.go.kr/policy-information/environment-energy/air-quality-information/1-air-quality-information/. The airKorea site does have an english version, though it appears to be mostly flash. However, they do also have a CSV download for hourly trends. http://www.airkorea.or.kr/eng/search/hourlyTrends
as an aside, http://thedailyviz.com/2016/06/03/charting-air-pollution/#more-2230 Matt Stiles may have insights on where to pull good data sets for air for korea.
Information from a community member re: South Korean data (overlaps some with the above):
South Korea’s hourly airquality data can be found here: airkorea.or.kr Some part of the page is in English, too. Eng.airkorea.or.kr Today I have some problems with Adobe Flash. Not all graphs, show, but you’ll know that this is the right place.
This page has past results that have been validated: http://www.airkorea.or.kr/pastSearch
Old data and reports back to 1998: http://www.airkorea.or.kr/detailViewDown
"Note that this data is prior to approval of use."
Do we need permission to scrape here?
Our modus operandi to date has been to assume that all air quality data shared on a public forum is okay for ingestion into the OpenAQ system. If a data provider does request we take down these data, we will (though also to note, that hasn't happened to date). For extra measure, I am emailing the privacy officer listed on the site to let them know we are working on ingesting their data and to let us know if there are any issues with this. Thanks, @dolugen!
AirKorea has an API called OpenAPI. It says that you need to register an account at the "open data portal" or www.data.go.kr, which is the Korean government's open data portal -- similar to our Data.gov site, to gain access to their API. ~I am having trouble accessing www.data.go.kr to find out more info.~
Information about AirKorea's Open API on www.data.go.kr. The service is free (need to register, though). The documentation is in Korean, but I think a seasoned developer with basic AQ knowledge can make a good educated guess on what's what, but I can help with translation.
Attaching the API documentation (in Korean) likely linked above but also recently shared by a community member. airkorea_openapi_guide-v1_6_1.docx
Questions I have about the docs (and am asking around for answers, but chime in if anyone already on this chain knows):
- I can't tell if the stations have geo coords.
- Is time-averaging indicated? e.g. is the data hourly?
- What are the different 'grades'? Is one a "real-time' layer? We don't want to accidentally mesh in QA/QC'ed data on delay with the rest of the real-time gov't layer.
If these things aren't in place, I'm planning on reaching out to two groups to see if we can get the info.
cc: @mondorescue, in case still of interest. And I just re-realized that this is you, Brian!!
@RocketD0g I can help out with this.
From the documentation, there is no information whether the stations have geo coords -- probably need to look them up elsewhere.
The 'grades' are operation numbers. (1) is for real-time data from each measurement station for a select period (daily, monthly, or quarterly). -- Doesn't say what the time-averaging is. (2) is for listing locations with "poor" air quality index rating. (3) is for real-time measurement by "city road" [not 100% sure how to translate that. It is like a city "ward", "neighborhood", or "block"]. (4) is for air quality forecast of PM10 and Ozone. (5) is for real-time data by "city road" of pollutants (PM10, SO2, CO, O3, NO2, etc.) by select time-averaging (hourly, daily, etc.) for a select period (in weeks, months, etc.). -- I'll have to grab some sample data to really understand what they mean... (6) is for real-time data by city-district for a select period (hourly, daily, etc.)
I'm having a bit trouble grabbing some sample data, because the OpenAPI website requires a special web browser add-on (and it only works for IE 9+) to function. Once I sort this out, I'll let you know if there are any geo coords info or where to get that.
@RocketD0g From API documentation-same version as attached above. -For each station, there is a station name, address, and latitude/longitude. -I'm trying to verify if data is collected hourly. Seems like it from what I'm reading. **Might be easier to ask in person.
From API website: -"real-time data before authentication"
@RocketD0g From API documentation, I think we want the values (PM2.5 value, O3 value); that's the measured density with the proper units. The grade is the pollution rating or index.
The confusing thing is pm25value24, that is literally translated as, "24-hour prediction transfer concentration". Why is thing named similarly to pm25value, when it is likely calculated?
Thanks @AreteY and @mondorescue. This is helpful. Basically, I'm hoping to we can contact a colleague to ask whatever gaps remain in our access. Sounds like our main q at the moment is time averaging.
-For each station, there is a station name, address, and latitude/longitude.
I wasn't able to find this information in the API documentation, but there is a full list of stations with their id and lat/lon in Airkorea's monthly atmospheric report (see example, here starting page 95).
-I'm trying to verify if data is collected hourly. Seems like it from what I'm reading. **Might be easier to ask in person.
In the monthly atmospheric report starting on page 108, it describes how the stats are calculated and it appears measurements are hourly averages. In the annual atmospheric report (e.g. "2018 대기환경연보") on page 9, it's list how measurements are made for the pollutants, and it says measurements are taken at "1-hour continuous" (or hourly averages).
@AreteY @mondorescue I was taking a look at adding this data source just now, but I've poked around for a bit on https://www.data.go.kr/ and can not find a way to create an account. Is there any way you could point me to where that can be done? Thanks!
@jflasher You can visit this link (note, the entire registration process is in Korean). The image on the left with the orange bar is registration for individuals. The image on the right with the "green" bar is for institutions.
Hi All- Here's an update on what I've learned about the S. Korea AQ data. Their data is posted on this site http://www.airkorea.or.kr/web and they have an OpenAPI https://www.data.go.kr/dataset/15000581/openapi.do . They measure SO2, CO, O3, NO2, PM10, PM2.5 with the proper units (ppm, ug/m3). The information about their stations including their lat/long coordinates (they call dmX, dmY) can be retrieved using the Measurement Station Inquiry Service (service ID: SC-OA-09-01; MsrstnInfoInqireSvc) from their API. The AQ data can be retrieved using the Air Pollution Info Inquiry Service (service ID: SC-OA-09-02; ArpltnInforInqireSvc) from their API. The desired outputs are called so2Value, coValue, o3Value, no2Value, pm10Value, pm25Value, and these are updated every 10 minutes (rolling hourly average every 20 minutes). The reference for all of this is the OpenAPI Guide v1.7.2. Lastly I believe @mondorescue has a "personal" API key; there's a thread on the Slack #dev channel about this with @Brian about this. He mentions we can get a "company" key, too.
For station inquiry, the URL endpoint is: http://openapi.airkorea.or.kr/openapi/services/rest/MsrstnInfoInqireSvc/getMsrstnList For AQ data inquiry, the URL endpoint is: http://openapi.airkorea.or.kr/openapi/services/rest/ArpltnInfoInqireSvc/getMsrstnAcctoRltmMesureDnsty
Hi @mondorescue, I tried to register for a company api key from the registration link you posted, but got an error page, mind posting the link again? And any pointers for registration since it's in Korean?
@sruti The website has completely changed since March. Here's the new link. I will have to go through the registration process using another email address to find out what's changed, but the personal API key I got seems to still work, which I don't mine sharing with the OpenAQ team for dev work.
Thanks so much @mondorescue! Was able to go through registration and get a key for OpenAQ.
Having access trouble with this request to get station information (with the service key filled in) http://openapi.airkorea.or.kr/openapi/services/rest/MsrstnInfoInqireSvc/getMsrstnList?addr=%EC%84%9C%EC%9A%B8&stationName=%EC%A2%85%EB%A1%9C%EA%B5%AC&pageNo=1&numOfRows=10&serviceKey= (with the service key filled in)
We get back
<response>
<header>
<resultCode>20</resultCode>
<resultMsg>SERVICE ACCESS DENIED ERROR. Please ask data.go.kr</resultMsg>
</header>
</response>
But this request works perfectly well:
http://openapi.airkorea.or.kr/openapi/services/rest/ArpltnInforInqireSvc/getMsrstnAcctoRltmMesureDnsty?serviceKey=&numOfRows=10&pageNo=1&stationName=%EC%A2%85%EB%A1%9C%EA%B5%AC&dataTerm=DAILY&ver=1.3&
@mondorescue @AreteY would you happen to have any insight from the documentation about why this might be?
I'm going to email them as well.
@sruti Have the Open API folks replied back to you? Scanning the Open API data portal forums, the Open API folks might be having some sort of server problem since July (or end of June). A number of devs on the forums are asking about the Code 20 SERVICE ACCESS DENIED ERROR... If I find anything else, I'll report back here.
Hi @mondorescue thanks for looking into it! Sorry forgot to post an update here. They did get back to me. It turns out you need to separately apply for permission to use theMsrstnInfoInqireSvc endpoint, though the same API key can be used for both.
- Updated AirKorea site
- Updated Korea public data portal
- This api has station names and parameter values for the yuseong district, does not require API key.
- The data portal does not appear to have an API for all of Korea, but a few separate ones.
- Needs further investigation
