Work out and document how to integrate `capnp compile` into the cabal build process.

[zenhack]

It would be nice to have clear docs on how to have your schema files processed automatically at build time. For most projects, a good way to do this would be via a custom setup script that calls the command line tool. I'd like us to have a fleshed out example of this as documentation; maybe we can use it in the examples/ directory. Maybe we can also use this for building the stuff only used in tests.

Note that we can't do this for the main library, since the code generator itself depends its output -- so we're stuck with the generated code being committed.

[p4l1ly]

I've come up with the following Setup.hs script, it might be helpful to you or people looking for some "how to" until the example gets settled down:

{-# LANGUAGE LambdaCase #-}

import Distribution.Simple (defaultMainWithHooks, simpleUserHooks, UserHooks(preBuild))
import Distribution.Simple.Setup (BuildFlags(buildDistPref), fromFlag)
import Distribution.PackageDescription (BuildInfo(hsSourceDirs), emptyBuildInfo)
import System.Directory (createDirectoryIfMissing, findExecutable)
import System.Process (callProcess)
import System.Exit (die)

main = defaultMainWithHooks simpleUserHooks
  { preBuild = \args buildFlags -> do
      capnp <- findExecutable "capnp" >>= \case
        Just path -> return path
        Nothing -> die "setup: Could not find executable capnp"

      let gensrc = fromFlag (buildDistPref buildFlags) ++ "/gensrc"
      createDirectoryIfMissing False gensrc
      callProcess capnp ["compile", "-ohaskell:" ++ gensrc, "schema/afa.capnp"]

      return (Just emptyBuildInfo{hsSourceDirs=[gensrc]}, [])
  }

and the following relevant parts of the .cabal file:

extra-source-files:  CHANGELOG.md, README.md, schema/afa.capnp
build-type: Custom

library
  exposed-modules:     MyLib
  autogen-modules:
    Capnp.Gen.ById.Xf7c01c6f70140b00
    Capnp.Gen.Schema.Afa
    Capnp.Gen.Schema.Afa.Pure
  other-modules:
    Capnp.Gen.ById.Xf7c01c6f70140b00
    Capnp.Gen.Schema.Afa
    Capnp.Gen.Schema.Afa.Pure
  build-tool-depends:
    capnp:capnpc-haskell ^>= 0.6.0.0,
  build-depends:
    base ^>=4.13.0.0,
    capnp ^>=0.6.0,
  hs-source-dirs:      src/hs
  default-language:    Haskell2010

custom-setup
  setup-depends:
    Cabal ^>=3.2.0,
    base ^>=4.13.0.0,
    process ^>=1.6.8.0,
    directory ^>=1.3.6.0,

[zenhack]

Thanks for doing a pass on this.

One thing that's kindof unfortunate about this approach is that because it re-writes the output files every time, it won't successfully cache the build for the capnp modules; it will rebuild them from scratch each time you do cabal build. It would be nice to find a way to avoid that.

[p4l1ly]

You're right, the build times started freaking me out, so I have added the following imports to Setup.hs:

import Control.Monad (when)
import Distribution.Compat.Time (getModTime)

and the following changes to the preBuild code (in Setup.hs):

-       callProcess capnp ["compile", "-ohaskell:" ++ gensrc, "schema/afa.capnp"]
+       modTime_gen <- getModTime$ gensrc ++ "/Capnp/Gen/Schema/Afa.hs"
+       modTime_schema <- getModTime "schema/afa.capnp"
+       when (modTime_gen < modTime_schema)$
+         callProcess capnp ["compile", "-ohaskell:" ++ gensrc, "schema/afa.capnp"]

The build times are better, not sure if ideal.

EDIT removed the unix dependency

[zenhack]

Hm, I just tried adapting your change to https://github.com/zenhack/ocap-merkledag, and when I do:

touch schema/protocol.capnp
cabal build

...it doesn't rebuild anything. Same thing if I actually make substantive changes to the file (adding a struct or something). Is correctly picking up changes for you?

Also, will this work correctly if the generated code doesn't already exist? I would expect getModTime to throw an exception in that case.

[p4l1ly]

Yes, it is picking the changes correctly for me (even with touch). And yes, sorry, I've forgot about the initial case when the file does not exist. This is the corrected version:

{-# LANGUAGE LambdaCase #-}

import Control.Monad (when)
import Distribution.Compat.Time (getModTime)
import Distribution.Simple (defaultMainWithHooks, simpleUserHooks, UserHooks(preBuild))
import Distribution.Simple.Setup (BuildFlags(buildDistPref), fromFlag)
import Distribution.PackageDescription (BuildInfo(hsSourceDirs), emptyBuildInfo)
import System.Directory (createDirectoryIfMissing, findExecutable, doesFileExist)
import System.Process (callProcess)
import System.Exit (die)

main = defaultMainWithHooks simpleUserHooks
  { preBuild = \args buildFlags -> do
      capnp <- findExecutable "capnp" >>= \case
        Just path -> return path
        Nothing -> die "setup: Could not find executable capnp"

      let gensrc = fromFlag (buildDistPref buildFlags) ++ "/gensrc"
      createDirectoryIfMissing False gensrc

      let oneOfGeneratedFiles = gensrc ++ "/Capnp/Gen/Schema/Afa.hs"
      regenerate <- doesFileExist oneOfGeneratedFiles >>= \case
        True -> (<) <$> getModTime oneOfGeneratedFiles <*> getModTime "schema/afa.capnp"
        False -> return True

      when regenerate$
        callProcess capnp ["compile", "-ohaskell:" ++ gensrc, "schema/afa.capnp"]

      return (Just emptyBuildInfo{hsSourceDirs=[gensrc]}, [])
  }

[p4l1ly]

in case it makes any difference:

% cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library

% ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.3

Running on Linux.

[zenhack]

Ok, there's probably something else wrong on my end then; I'll have to troubleshoot at some point.

...it actually might be nice to package this up as its own small library (once we've worked out the kinks) so you can just do e.g:

-- Setup.hs
import Capnp.CabalSetup (defaultMain)
main = defaultMain