aeson icon indicating copy to clipboard operation
aeson copied to clipboard

UTCTime is not encoded according to tutorial

Open zeha opened this issue 7 years ago • 6 comments

Tutorial on https://artyom.me/aeson claims this on UTCTime:

UTCTime and ZonedTime are represented as strings according to the ISO 8601 standard (specifically, the ECMA-262 subset), which every other JSON parser should understand as well. You can also wrap a date into DotNetTime to get the non-standard format used by Microsoft.

The linked ECMA-262 page then has to say:

The format is as follows: YYYY-MM-DDTHH:mm:ss.sssZ

and

sss is the number of complete milliseconds since the start of the second as three decimal digits.

However, Aeson will encode the sub-second part with either three, six, or twelve digits. https://github.com/bos/aeson/blob/6e57d28/Data/Aeson/Encoding/Builder.hs#L194-L221

With an Android parser on the other side this leads to really annoying interop issues. Settling on the three-digit format would be good for interop.

zeha avatar Jan 16 '18 12:01 zeha

Hey,

The linked tutorial is not the official documentation, is it just a mistake in there or did aeson previously commit to this? Does ISO 8601 put limits on the precision? That said, it may make sense to change it, I'm torn on the issue so I'd appreciate input from others.

One one hand it does make sense to allow for all of the available precision in the time package, which is 12 digits. I've needed more than 3 digits in the past, and simply changing this may silently break any code I have depending on that.

But on the other hand it may be problematic that e.g. java.time.Instant only parses up to 9 digits, and JavaScript simply truncates anything after the first 3.

Which is the "android parser" you are referring to?

For now there is a workaround: Create a newtype with an encoder that truncates down to the number of digits you want. This could also be provided by aeson for the alternative no matter how we choose to proceed on this issue.

bergmark avatar Jan 16 '18 13:01 bergmark

Rgd. android parser: actually this is SimpleDateFormat (https://developer.android.com/reference/java/text/SimpleDateFormat.html), which requires that you know in advance how many digits the sub-second part has. So it could do 6 or 12 digits too, but then they have to be there all the time - effectively this means truncating trailing zeroes doesn't work.

zeha avatar Jan 16 '18 14:01 zeha

I can't find any actual formatting in the official docs, so maybe there are no commitments. For interop reasons, a standard 3-digit / millisecond-only format would still be great to have.

As I see it, the JavaScript/ECMA262 Date.parse() method also only supports millisecond resolution.

zeha avatar Jan 16 '18 14:01 zeha

Not sure about the spec, but in practice:

> Date.parse("2017-01-01T00:00:00.1Z")
1483228800100
> Date.parse("2017-01-01T00:00:00.123456Z")
1483228800123

bergmark avatar Jan 16 '18 15:01 bergmark

FWIW, we have https://hackage.haskell.org/package/aeson-1.2.3.0/docs/Data-Aeson-Types.html#t:DotNetTime, so we can have WhateverSpecTime. ECMA-262 is EcmaScript standard, not JSONs. I'm strongly for supporting full range of the type in the default instance, so roundtrip property holds.

phadej avatar Jan 16 '18 15:01 phadej

+100 to @phadej JavaScript has nothing to do with this.

llelf avatar Mar 04 '18 22:03 llelf

FWIW, SimpleDataFormat is old java.util.Date stuff. Don't use it, even on android.

I don't think there is anything to do for this issue.

phadej avatar Jun 19 '23 12:06 phadej