Mismatch in format used by elm-bridge and Aeson

[ahri]

My setup is that I have a websocket server running in Haskell, accepting commands encoded into JSON by elm-bridge's encoders on the client-side, and decoded by Aeson on the server-side.

On the server, in Haskell I have my data types defined as follows:

{-# LANGUAGE DeriveGeneric, DeriveAnyClass #-}

module Domain.Data where

import GHC.Generics
import Data.Aeson

data WebSocketMessage = WebSocketMessage String deriving (Eq, Show, Generic, ToJSON, FromJSON)
data WebSocketCommand = WebSocketCommand Command deriving (Eq, Show, Generic, ToJSON, FromJSON)

data Id = Id String deriving (Eq, Show, Ord, Generic, ToJSON, FromJSON)
data EditDone = EditDone Id DoneState DoneState deriving (Eq, Show, Generic, ToJSON, FromJSON)
data DoneState   = Done | NotDone deriving (Eq, Show, Generic, ToJSON, FromJSON)

data Command
    = EditDoneCommand EditDone
    | OtherCommands

and these are decoded in the server via:

    let mCmd :: Maybe WebSocketCommand = Json.decode msg

Note that I'm using the Generic, ToJSON, and FromJSON automatic derivations for the server-side decoding via Aeson.

Also in Haskell I'm executing elm-bridge, like so:

{-# LANGUAGE TemplateHaskell #-}
import Elm.Derive
import Elm.Module

import Data.Proxy

import Domain.Data

deriveElmDef defaultOptions ''EditDone
deriveElmDef defaultOptions ''Command
deriveElmDef defaultOptions ''Id
deriveElmDef defaultOptions ''DoneState

main :: IO ()
main = do
    putStrLn $ makeElmModule "Domain.Data"
        [ DefineElm (Proxy :: Proxy EditDone)
        , DefineElm (Proxy :: Proxy Command)
        , DefineElm (Proxy :: Proxy Id)
        , DefineElm (Proxy :: Proxy DoneState)

Note that I'm only running "deriveElmDef" as the Aeson derivations are already done separately. This took a bit of working out on my part as I don't think it's documented, so it may well be that I got this bit wrong; I couldn't get my program to compile any other way though!

Right, now to the meat; I added some tracing to my server and I can see that I receive JSON in the form:

{"EditDoneCommand":["foo","NotDone","Done"]}

But when Aeson decodes it I get a Nothing. So I added an output for Aeson's encoding, and it turns out that Aeson outputs a different format:

{"tag":"EditDoneCommand","contents":["foo","NotDone","Done"]}

Which looks like a longer form.

Given that I'm not familiar with Aeson, have I misconfigured something in my enthusiasm to get the thing compiling? Do you know how I can remedy this situation?

Thanks in advance for any advice!

PS. I have elm-bridge 0.4.0 and Aeson 0.11.2.1 installed.

[agrafix]

This looks like a mismatch of the aeson Options, the defaultOptions of elm-bridge are not the same defaultOptions as in aeson (anymore?)... You can fix it by writing something like this:

import Data.Aeson.Types (defaultTaggedObject)

deriveElmDef (defaultOptions { sumEncoding = defaultTaggedObject } ''DoneState

cc @bartavelle : Any thoughts?

[ahri]

Thanks for the rapid response, I'll need to change a few things to test this out so it'll have to be on the train home from work this evening.

Out of interest how can I do the opposite? i.e. change the way Aeson is generating its decoders. I'd like to weigh up whether it's worth doing it that way around instead but I wonder whether this will preclude me from using automatic derivations in my Haskell code.

[agrafix]

Yes, you can. See https://hackage.haskell.org/package/aeson-1.2.1.0/docs/Data-Aeson.html#v:genericToJSON

[bartavelle]

@agrafix you found the problem. The documentation focuses on deriving both Elm and Aeson code at the same time, so that there is no mismatch between the two representations. I chose to alter the defaultOptions so that it is easier to work with in javascript.

The defaultOptions in elm-bridge is defined here in case you want to alter the aeson side.

[ahri]

Interesting. I'm not sure where I stand on the preference actually - I think I like having the "tag" in there in some ways.

The trouble with altering Aeson's configuration is that I lose the ability to say

data Id = Id String deriving (Eq, Show, Ord, Generic, ToJSON, FromJSON)

• I'm lazy and quite like the magic "deriving" stuff!

Having said that I'll have to mess around a lot with my JS stuff to support the tag/content format :(

[bartavelle]

If you like aesons's preferences, then you can use them in elm-bridge!

[ahri]

Interestingly if I use

deriveElmDef (defaultOptions { sumEncoding = defaultTaggedObject }) ''DoneState

Then invalid Elm is generated:

jsonDecEditDone : Json.Decode.Decoder ( EditDone )
jsonDecEditDone =
    Json.Decode.map3 EditDone (Json.Decode.index 0 (jsonDecId)) (Json.Decode.index 1 (jsonDecDoneState)) (Json.Decode.index 2 (jsonDecDoneState))
        jsonDecObjectSetEditDone = Set.fromList []

Note the last line, I'm not sure what it's trying to do there.

This is on 0.4.1 (I just upgraded).

[ahri]

After messing around with the Elm file manually, deleting those spurious cases lines, it now works, so that's a strange bug.

[bartavelle]

I can't reproduce this problem, could you attach a whole haskell module that demonstrates the problem?

(Also Id should be a newtype).

[ahri]

I actually moved away from newtype because an earlier version of elm-bridge didn't like it - I forget why, but I've been ignoring the linter since then ;) I'll swap back to newtype!

I'll also endeavour to replicate the problem I mentioned properly.

[agrafix]

Any updates on this @ahri ? Maybe you can contribute a test case?

[ahri]

Sorry, I've been quite busy :\

Cabal file extract:

build-type:          Simple
cabal-version:       >=1.10

executable elmbridge
  hs-source-dirs:      elmbridge
  main-is:             Main.hs
  default-language:    Haskell2010
  build-depends:       base >= 4.7 && < 5
                     , elm-bridge >= 0.4.1 && < 0.5.0
                     , aeson >= 0.11.2.1 && < 0.12
                     , test

library
  hs-source-dirs:      src
  default-language:    Haskell2010
  exposed-modules:     Test.Data
  ghc-options:         -Wall -Werror
  build-depends:       base >= 4.7 && < 5
                     , containers >= 0.5.7.1 && < 1
                     , aeson >= 0.11.2.1 && < 0.12

stack.yml contains:

extra-deps: [elm-bridge-0.4.1]

{-# LANGUAGE TemplateHaskell #-}

module Test.Data where

import Data.Aeson.TH (deriveJSON, defaultOptions)

data Id = Id String deriving (Eq, Show, Ord)
data Foo = Foo Id (Id, Id)

deriveJSON defaultOptions ''Id
deriveJSON defaultOptions ''Foo

Main:

{-# LANGUAGE TemplateHaskell #-}

module Main where

import Elm.Derive
import Elm.Module

import Data.Proxy
import qualified Data.Aeson.TH as TH

import Test.Data

deriveElmDef (TH.defaultOptions { sumEncoding = TH.defaultTaggedObject }) ''Id
deriveElmDef (TH.defaultOptions { sumEncoding = TH.defaultTaggedObject }) ''Foo

main :: IO ()
main = do
    putStrLn $ makeElmModule "Test.Data"
        [ DefineElm (Proxy :: Proxy Id)
        , DefineElm (Proxy :: Proxy Foo)
        ]

Also, if I change Id to a newtype (as in only changing "data Id" to "newtype Id"), per @bartavelle's note, I get a compilation error:

Main.hs:13:1: error:
    Oops, can only derive data and newtype, not this: NewtypeD [] Test.Data.Id [] Nothing (NormalC Test.Data.Id [(Bang NoSourceUnpackedness NoSourceStrictness,ConT GHC.Base.String)]) []

[bartavelle]

Cool, two bugs for the price of one :) I will work on them next week.

[bartavelle]

Btw, the newtype problem should be fixed now.

[bartavelle]

Oh well, it only took me 6 month to work on these :(