siphon icon indicating copy to clipboard operation
siphon copied to clipboard

Add IEM Surface Data

Open jrleeman opened this issue 8 years ago • 6 comments

Be able to request surface data from the IA State archive and hide multiple requests behind a convenient interface.

jrleeman avatar Sep 20 '17 14:09 jrleeman

Is this actively being worked on? I'd be happy to put something together if nobody is.

WEP11 avatar Feb 23 '18 15:02 WEP11

Have at it! I believe @akrherz was open to adapting the REST API for the service if we found something lacking.

dopplershift avatar Feb 23 '18 20:02 dopplershift

I'll take a look at it next week! It's related to a future project we're exploring here, so I'll put it on my 'official' to-do list.

WEP11 avatar Feb 23 '18 20:02 WEP11

IemAsos(stationList, startDate, endDate)

What I have so far will build a URL using https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py and parse the results. Right now it parses the data into a chronological list, very much like the raw output you get in the first place. Thus this example would give the following results:

Python:

start = datetime(2018,1,1)
end = datetime(2018,1,2)
asos = IemAsos(['KLNK', 'KGSO'], start, end)

print(asos.headers)
print(asos.asosData[0])
print(asos.asosData[1])

Output:

['station', 'valid', 'lon', 'lat', 'tmpf', 'dwpf', 'relh', 'drct', 'sknt', 'p01i', 'alti', 'mslp', 'vsby', 'gust', 'skyc1', 'skyc2', 'skyc3', 'skyc4', 'skyl1', 'skyl2', 'skyl3', 'skyl4', 'wxcodes', 'metar']
['LNK', datetime.datetime(2018, 1, 1, 0, 0), -96.7647, 40.8478, None, None, None, 10.0, 7.0, None, 30.84, None, 10.0, None, 'CLR', 'M', 'M', 'M', 'M', 'M', 'M', 'M', 'M', 'KLNK 010000Z AUTO 01007KT 10SM CLR M21/M26 A3084 RMK T12101260 MADISHF']
['GSO', datetime.datetime(2018, 1, 1, 0, 0), -79.9436, 36.0975, None, None, None, 30.0, 8.0, None, 30.29, None, 10.0, None, 'OVC', 'M', 'M', 'M', '5000.00', 'M', 'M', 'M', 'M', 'KGSO 010000Z AUTO 03008KT 10SM OVC050 M05/M22 A3029 RMK T10501220 MADISHF']

The parser is:

  1. Converting the date strings to datetime objects
  2. Converting non-text variables to float values (or None if missing)
  3. Storing the headers as a separate variable for reference

Remaining questions in order of importance (I think):

  • Should the individual stations be split into different lists? (I imagine so)
  • Do we want the user to be able to use the data in a dictionary-like manner rather than knowing the index of their variable?
  • Am I missing something major here? (Wrong data source or something)

I'm sorry I've waited 8 months to start tackling this!

WEP11 avatar Oct 25 '18 21:10 WEP11

No worries! This looks like a nice start. The biggest things I would want to see is that this return things as a Pandas DataFrame, rather than doing things with lists/separating stations.

dopplershift avatar Oct 25 '18 21:10 dopplershift

That would definitely be more comfortable! I'll get it added!

WEP11 avatar Oct 25 '18 21:10 WEP11