Experience report: Upgrading from 0.4.0.0 to 0.5.1.0

[gasi]

NOTE: This is an experience report, not a bug report / feature request. I couldn’t find any community or support channel, so I figured I’d share here. Feel free to close/move if there is a better place for it.

First of all: Thanks for the wonderful squeal library. It gives me a fuzzy feeling to be able to write composable and type-safe SQL for my web app ZoomHub. I think it does a good job highlighting some of the possibilities of adopting Haskell ❤️

I just wrapped up an afternoon upgrading my app from Squeal 0.4.0.0 to 0.5.1.0. Overall, I really like the improvements, e.g.

• Query_ and Manipulation_ abstracting TuplePG and RowPG
• Omission of Same values. AFAICT, this is a huge improvement over other libraries I’ve used in the past, e.g. opaleye (though they might have caught up with this):

-incrNumViews :: Manipulation Schema '[ 'NotNull 'PGint8, 'NotNull 'PGtext ] '[]
+incrNumViews :: Manipulation_ Schemas (Int64, Text) ()
 incrNumViews =
   update_ #content
-    ( Same `as` #id :*
-      Same `as` #hash_id :*
-      Same `as` #type_id :*
-      Same `as` #url :*
-      Same `as` #state :*
-      Same `as` #initialized_at :*
-      Same `as` #active_at :*
-      Same `as` #completed_at :*
-      Same `as` #title :*
-      Same `as` #attribution_text :*
-      Same `as` #attribution_link :*
-      Same `as` #mime :*
-      Same `as` #size :*
-      Same `as` #error :*
-      Same `as` #progress :*
-      Same `as` #abuse_level_id :*
-      Same `as` #num_abuse_reports :*
-      Set ( #num_views + param @1 ) `as` #num_views :*
-      Same `as` #version
-    )
+    ( Set ( #num_views + (param @1) ) `as` #num_views )
     ( #hash_id .== param @2 )

• insertInto replacing insertRow and insertQuery
• Support for literal values. Before, I had to hack it using an unwieldy Expression
• Improvements to migrations, e.g. pureMigration, etc.

Here’s the link that outlines how to upgrade from Squeal 0.4.0.0 to 0.5.1.0 for a non-trivial app: https://github.com/zoomhub/zoomhub/pull/131/commits/ab9ae0b4c8616b6d15ac149c6556a61aaa68374e

I used the release notes as well as the raw source to figure out what changes were needed. Updating the API usage was fairly straightforward. What took me longer is figuring out how to update certain type signatures, e.g.

 selectContentBy
-  :: ( TableExpression Schema '[ 'NotNull a ] _ 'Ungrouped ->
-       TableExpression Schema '[ 'NotNull a ] _ 'Ungrouped
+  :: ( TableExpression '[] '[] 'Ungrouped Schemas '[ 'NotNull a ] _ ->
+       TableExpression '[] '[] 'Ungrouped Schemas '[ 'NotNull a ] _
      )
-  -> Query Schema '[ 'NotNull a ] (RowPG ContentImageRow)
-selectContentBy clauses = select
+  -> Query '[] '[] Schemas '[ 'NotNull a ] (RowPG ContentImageRow)

and

 getBy ::
-  (MonadBaseControl IO m, MonadPQ Schema m, ToParam p a) =>
-  Condition Schema _ 'Ungrouped '[ 'NotNull a ] ->
+  (MonadUnliftIO m, MonadPQ Schemas m, ToParam p a) =>
+  Condition '[] '[] 'Ungrouped Schemas '[ 'NotNull a ] _ ->
   p ->
   m (Maybe Content)

I hope this is helpful for others who want to upgrade to the latest release.

P.S. Minor request: Would it be possible to git tag -a … the various releases? That would have helped me comparing code between different versions. Thanks 🙇

[echatav]

Thanks so much for this @gasi!

[echatav]

Do you know of a guide for git tagging? Sorry, not too familiar with it.

[echatav]

For the literal, are you not able to literally use a literal? :-)

e.g. "$placeholder-overwritten-by-trigger$" instead of literal ("$placeholder-overwritten-by-trigger$" :: Text)?

(with OverloadedStrings turned on)

[gasi]

Do you know of a guide for git tagging? Sorry, not too familiar with it.

Absolutely! Depending on whether you prefer git CLI or GitHub UI, you can use one of these:

• CLI: https://stackoverflow.com/a/18223354/125305
• UI: https://help.github.com/en/github/administering-a-repository/creating-releases

To tag previous releases:

Find roughly when version in Cabal file was bumped

> git log -- squeal-postgresql/squeal-postgresql.cabal
commit b82ac4f6d19a49935149ef7ced9dcf639b7e8c28
Author: Eitan Chatav <[email protected]>
Date:   Tue Jun 18 14:12:19 2019 -0700

    0.5.1.0

Tag commit

git tag --annotate --message='Release 0.5.1.0' 0.5.1.0 b82ac4f6d19a49935149ef7ced9dcf639b7e8c28

Push tag

git push origin 0.5.1.0

This will generate a new release here https://github.com/morphismtech/squeal/releases and will allow people to locally check out a particular version, e.g. git checkout 0.5.1.0, or even compare versions, e.g. git diff 0.4.0.0 0.5.1.0.

[gasi]

For the literal, are you not able to literally use a literal? :-)

Ha, indeed — thanks! Just fixed it: https://github.com/zoomhub/zoomhub/pull/131/commits/ab9ae0b4c8616b6d15ac149c6556a61aaa68374e#diff-ed4a782deea6e05bcc7afab1fab4ec8dR345

I have a feeling there are other things I am not doing optimally, but I figured I’ll ask you specifics once I ship this:

• https://github.com/zoomhub/zoomhub/blob/e677551412a90394572424e995b2b58a9b63d8ea/src/ZoomHub/Storage/PostgreSQL.hs
• https://github.com/zoomhub/zoomhub/blob/e677551412a90394572424e995b2b58a9b63d8ea/src/ZoomHub/Storage/PostgreSQL/Internal.hs

In the meantime, if anything stands out, feel free to ping me 😄

[echatav]

one thing I noticed, you used pureMigration bc your overall migrations were impure. But you may be able to keep all your migrations pure (in the sense of being just SQL, not any other IO) by using the manipDefinition (poorly named, got a better suggestion?) to cast your manipulations as definitions. Unless you're actually using IO in your migrations, I might have overlooked.

[gasi]

Awesome — thanks for the tip! You’re right, none of my migrations need IO. I simply never came across manipDefinition. This commit outlines the change: https://github.com/zoomhub/zoomhub/commit/504f57b779e8852feda6bdfbe7ed12a76be5e0cc

Re: naming of manipDefinition. Good question. My first instinct would be to avoid the abbreviation and name it manipulateDefinition (?). That would align it well with manipulate and would make it easier to search for. For now, this could be introduced as an alias and the original could get a deprecation warning. That way the library stays backwards-compatible.

Re: combining definitions. I have to admit, figuring out how to append multiple definitions took me some sleuthing to find the Category instance and the relevant part in the docs

Definitions may be composed using the >>> operator.

This is the first time I am using Category, but in this case it looks like Definition could also have a Monoid instance where mempty = Category.id and (<>) = (>>>). Would be open to a PR if that looks right to you? I think it would be a bit more beginner friendly and allow the use of mconcat to do [Definition schemas schemas] -> Definition schemas schemas:

https://github.com/zoomhub/zoomhub/blob/504f57b779e8852feda6bdfbe7ed12a76be5e0cc/src/ZoomHub/Storage/PostgreSQL/Schema.hs#L355-L358

---

BTW, I’ve been writing down questions/feedback for Squeal as I go and one item was:

• How to convert custom data type to Expression that can be used with

Set … `as` … :*

This has totally been solved by literal — thank you ❤️

[echatav]

it looks like Definition could also have a Monoid instance where mempty = Category.id and (<>) = (>>>). Would be open to a PR if that looks right to you?

Absolutely, the endomorphisms in any category always form a monoid and that should be true in Haskell too.

instance schemas0 ~ schemas1 => Semigroup (Definition schemas0 schemas1) where (<>) = (>>>)
instance schemas0 ~ schemas1 => Monoid (Definition schemas0 schemas1) where mempty = id

[echatav]

literals are getting some love in #169

[adfretlink]

Oh, thanks @gasi ! It's always cool to have some live Squeal code.

I support the idea of tagging. One of the plus side of using github release UI is that it lets you add some documentation for each release, so we could split the release notes for each release version.

[gasi]

Absolutely, the endomorphisms in any category always form a monoid and that should be true in Haskell too.

Sounds cool. I don’t understand category theory yet but thanks for sharing. I actually gave it a shot writing Semigroup and Monoid support for Definition. The code was easy but I struggled with the tests. I realized the reason I needed it was because I was cheating: I am adding a bunch of functions and triggers but pretending the schema goes from '[] to '[], e.g.

https://github.com/zoomhub/zoomhub/blob/2270c95a75f5181577369c6c7b680ccb0266f1c2/src/ZoomHub/Storage/PostgreSQL/Schema.hs#L169-L179

Based on my understanding of the compile errors, it seems like Monoids would have to be on the same schema which very rare. Even adding something small like a schema will change the type of the schema, turning the Monoid moot.

Re: literals love. Awesome — keep it up! ❤️

Update: I finally got my app working end-to-end using Squeal + PostgreSQL last night (after working on it sporadically for a couple of years). I’ll share it once it’s tested and stable. Thanks again for the awesome library and documentation 😄

[echatav]

I use Haskell basically so I can have an excuse to keep using category theory 😄 Hopefully, I'll be cutting an announcement for free-categories soon.