Propoal: operators similar to .= for Maybe or Functor/Applicative types

[jdreaver]

Oftentimes when manually writing ToJSON instances, we have fields with Maybe types and we want to just hide the field if its value is Nothing. I usually resort to something like this:

data A =
  A
  { x :: Int
  , y :: Maybe Int
  }

instance ToJSON A where
  toJSON (A x y) =
    object $ catMaybes
    [ Just ("x" .= x)
    , ("y" .=) <$> y
    ]

In my opinion, that is not very elegant. I propose adding the following two operators to make this nicer:

(.=!) :: (Applicative f, ToJSON v, KeyValue kv) => Text -> v -> f kv
k .=! v = pure $ k .= v
(.=?) :: (Functor f, ToJSON v, KeyValue kv) => Text -> f v -> f kv
k .=? v = (k .=) <$> v

I envision these operators becoming the duals of .:! and .:?, which are very useful when parsing. Using them would look like this:

instance ToJSON A where
  toJSON (A x y) =
    object $ catMaybes
    [ "x" .=! x
    , "y" .=? y
    ]

I'm still undecided if the general Functor/Applicative constraints are that useful, or if we should just specialize these operators to Maybe:

(.=!) :: (ToJSON v, KeyValue kv) => Text -> v -> Maybe kv
k .=! v = Just $ k .= v
(.=?) :: (ToJSON v, KeyValue kv) => Text -> Maybe v -> Maybe kv
k .=? v = (k .=) <$> v

Let me know what you think. I'm still a little hesitant on this proposal, mainly because I want to avoid needless operator soup. I thought it was a decent idea though, so I decided to make open this issue to at least open up discussion.

[jdreaver]

Also, if you want to see some real-world code that would benefit, I have tons of ToJSON declarations like this in one of my libraries: https://github.com/frontrowed/stratosphere/blob/7caca3b2994fedeaf1e68139892b6cc3a6747d64/library-gen/Stratosphere/Resources/EC2Instance.hs#L57-L90

[bergmark]

Regardless of these operators, have you considered deriving these instances? I avoid writing instances manually as much as possible, it's too cumbersome and too error-prone. What if you make a typo? What if you forget a field e.g. when adding a new one? Existing implementations for aeson may suit your needs. But in general learning generics (or TH) takes a bit of time but opening @kosmikus' box is well worthwhile: https://www.youtube.com/watch?v=sQxH349HOik

Another improvement you can make is to just declare these operators locally.

I'm always vary of adding operators as they are not self documenting and hard to learn and remember.

Mnemonics can be useful; aeson already has .=, .:, .:!, and .:?v .=?would fit into that, but not.=!. [The fact that .=?` it already exists elsewhere](https://hackage.haskell.org/package/aeson-utils-0.3.0.2/docs/Data-Aeson-Utils.html#v:.-61--63-) (written by me no less...) could be used to argue both that the name seems obvious enough for inclusion in aeson or that it's not needed because it already exists.

Am I correct in that the only reason for .=! is to be a .= for usage together with .=?? If so then i'd ideally like to reuse .=, although trying to fit that into the type class might not be worth the trouble.

I think adding .=? is a good idea. I would like to hear if there are other ideas for solutions to the problem .=? would solve. Overall I think we should have a much stronger focus on generics when in our documentation, I've thought about that before so I'll create a new issue out of it.

As for using Maybe or generalizing the types I'd lean towards using Maybe since it makes their usage more obvious and easier to understand.

As always I appreciate input from others in these things!

[jdreaver]

Regardless of these operators, have you considered deriving these instances?

Definitely, I try as much as possible to use TH or Generic to derive instances. In some cases, though, you have to write them manually. For the example I linked, that code is auto-generated, and I just write out the full instance to avoid any TH or Generic overhead.

Another improvement you can make is to just declare these operators locally.

Indeed, that's how these were born :smile:

Am I correct in that the only reason for .=! is to be a .= for usage together with .=??

Correct. Hmm, I can't think of a straightforward way to modify .= to work with .=? as well as work as it does currently. I wonder if .=? could be modified to fit both cases.

---

I'll have to sit on this a bit more to think if it's a good idea. I agree that .=? probably fits into the current operator taxonomy better.

Thanks for taking a look at this!

[tfausak]

An alternative to introducing new operators is to add a function that removes keys that point to null values. I came up with this:

import qualified Data.Aeson as Aeson
import qualified Data.HashMap.Strict as HashMap
import qualified Data.Vector as Vector

removeNullObjectValues :: Aeson.Value -> Aeson.Value
removeNullObjectValues value =
  let isNotNull v = case v of { Aeson.Null -> False; _ -> True }
  in case value of
    Aeson.Array array -> Aeson.Array $
      Vector.map removeNullObjectValues array
    Aeson.Object object -> Aeson.Object .
      HashMap.map removeNullObjectValues $ HashMap.filter isNotNull object
    _ -> value

Example usage:

>>> removeNullObjectValues [aesonQQ| { "k": null } |]
Object (fromList [])
>>> removeNullObjectValues [aesonQQ| { "k": "v" } |]
Object (fromList [("k",String "v")])

It would allow you to define instances like this:

instance ToJSON A where
  toJSON (A x y) =
    removeNullObjectValues . object $
    [ "x" .= x
    , "y" .= y
    ]