Timeseries data format

[wcarthur]

When running a simulation and extracting time series data for stations, the formatting of the datetime string means the interval is not equal, because machine precision seems to result in slightly varying intervals. When writing the data to file, we truncate (not round) the time field to minutes.

Solution:

• Expand the time format to "%Y-%d-%m %H:%M:%S"
• Do some rounding of the time values to eliminate the fractional seconds that is leading to this behaviour

[wcarthur]

Function definition for rounding datetime objects

def roundTime(dt=None, roundTo=60):
   "''Round a datetime object to any time lapse in seconds
   dt : datetime.datetime object, default now.
   roundTo : Closest number of seconds to round to, default 1 minute.
   Author: Thierry Husson 2012 - Use it as you want but don't blame me.
   """
   if dt == None : dt = datetime.datetime.now()
   seconds = (dt.replace(tzinfo=None) - dt.min).seconds
   rounding = (seconds+roundTo/2) // roundTo * roundTo
   return dt + datetime.timedelta(0,rounding-seconds,-dt.microsecond)

Needs unittest around it, but should be sufficient

[wcarthur]

Another option is to store the data as a collection of pandas.DataFrames. Then can use DataFrame.to_csv() with the date_format arg set to "%Y-%d-%m %H:%M:%S"