Make reproducible OCaml builds

[wiredsister]

• figure out how to commit transitive deps in Dune and restore from that
• pin packages to versions

[yanlow]

OCAML supports package version pinning like so: otuser@090b6c831cd0:/OpenTransplant$ opam pin utop 2.6.0

Turns out, pin information is stored here and is tied to your local switch configuration

otuser@090b6c831cd0:/OpenTransplant$ cat ~/.opam/ocaml-system.4.05.0/.opam-switch/switch-state
opam-version: "2.0"
compiler: [
  "base-bigarray.base"
  "base-threads.base"
  "base-unix.base"
  "ocaml.4.05.0"
  "ocaml-config.1"
  "ocaml-system.4.05.0"
]
roots: [
  "calendar.2.04"
  "cohttp-lwt-unix.2.5.4"
  "dune.2.7.1"
  "fmt.0.8.9"
  "js_of_ocaml.3.7.1"
  "js_of_ocaml-ppx.3.7.1"
  "logs.0.7.0"
  "merlin.3.4.2"
  "ocaml-system.4.05.0"
  "ocp-indent.1.8.1"
  "qcheck.0.16"
  "utop.2.6.0"
  "uuidm.0.9.7"
]
installed: [
  "angstrom.0.15.0"
  "astring.0.8.5"
  "base.v0.13.2"
  "base-bigarray.base"
  "base-bytes.base"
  "base-num.base"
  "base-threads.base"
  "base-unix.base"
  "base64.3.4.0"
  "bigarray-compat.1.0.0"
  "bigstringaf.0.7.0"
  "biniou.1.2.1"
  "calendar.2.04"
  "camomile.1.0.2"
  "charInfo_width.1.1.0"
  "cmdliner.1.0.4"
  "cohttp.2.5.4"
  "cohttp-lwt.2.5.4"
  "cohttp-lwt-unix.2.5.4"
  "conduit.2.0.2"
  "conduit-lwt.2.0.2"
  "conduit-lwt-unix.2.0.2"
  "conf-m4.1"
  "conf-pkg-config.1.3"
  "cppo.1.6.7"
  "csexp.1.3.2"
  "domain-name.0.3.0"
  "dot-merlin-reader.3.4.2"
  "dune.2.7.1"
  "dune-configurator.2.7.1"
  "easy-format.1.3.2"
  "fieldslib.v0.13.0"
  "fmt.0.8.9"
  "ipaddr.5.0.1"
  "ipaddr-sexp.5.0.1"
  "js_of_ocaml.3.7.1"
  "js_of_ocaml-compiler.3.7.1"
  "js_of_ocaml-ppx.3.7.1"
  "jsonm.1.0.1"
  "lambda-term.3.1.0"
  "logs.0.7.0"
  "lwt.5.4.0"
  "lwt_log.1.1.1"
  "lwt_react.1.1.4"
  "macaddr.5.0.1"
  "magic-mime.1.1.2"
  "menhir.20201216"
  "menhirLib.20201216"
  "menhirSdk.20201216"
  "merlin.3.4.2"
  "mew.0.1.0"
  "mew_vi.0.5.0"
  "mmap.1.1.0"
  "num.0"
  "ocaml.4.05.0"
  "ocaml-compiler-libs.v0.12.3"
  "ocaml-config.1"
  "ocaml-migrate-parsetree.1.8.0"
  "ocaml-secondary-compiler.4.08.1-1"
  "ocaml-syntax-shims.1.0.0"
  "ocaml-system.4.05.0"
  "ocamlbuild.0.14.0"
  "ocamlfind.1.8.1"
  "ocamlfind-secondary.1.8.1"
  "ocp-indent.1.8.1"
  "ocplib-endian.1.1"
  "ounit2.2.2.4"
  "parsexp.v0.13.0"
  "ppx_derivers.1.2.1"
  "ppx_fields_conv.v0.13.0"
  "ppx_sexp_conv.v0.13.0"
  "ppx_tools_versioned.5.4.0"
  "ppxlib.0.13.0"
  "qcheck.0.16"
  "qcheck-core.0.16"
  "qcheck-ounit.0.16"
  "re.1.9.0"
  "react.1.2.1"
  "result.1.5"
  "seq.0.2.2"
  "sexplib.v0.13.0"
  "sexplib0.v0.13.0"
  "stdio.v0.13.0"
  "stdlib-shims.0.1.0"
  "stringext.1.6.0"
  "topkg.1.0.3"
  "trie.1.0.0"
  "uchar.0.0.2"
  "uri.4.0.0"
  "uri-sexp.4.0.0"
  "utop.2.6.0"
  "uuidm.0.9.7"
  "uutf.1.0.2"
  "yojson.1.7.0"
  "zed.3.1.0"
]
pinned: "utop.2.6.0"

I think the most straightforward way to handle this is to take a known working configuration and install specific version numbers in the Dockerfile, and do an upgrade every now and then.

[wiredsister]

Yeah, @yanlow that sounds great to me.

We're kind of stuck because I can't cleanly upgrade opam on my machine for whatever reason and I'm also blocked on being able to find an actual lock file in my version 2.0.3. I'll do more digging and we can get this resolved. In the meantime, this is another vector of failure and security vulnerability. Adding a blocked label.

[wiredsister]

https://twitter.com/mjambon/status/1343736196522905603 👈 full answer from Martin about how to do dependency management with examples. Like I said, we avoid much drama with this issue through Dockerizing, but it's still worth solving.

[yawaramin]

Opam supports lockfiles, with exact version numbers of all direct and transitive dependencies. Here's a tutorial: https://khady.info/opam-sandbox.html (note, in latest opam versions opam lock is a built-in command, no longer a plugin).

The setup-ocaml GitHub action lets you set up a GitHub workflow with caching (fast builds): https://github.com/ocaml/setup-ocaml

One of its steps is: - run: opam install . --deps-only --with-test

When you have an opam lock file, you can change that to: - run: opam install . --locked --deps-only --with-test. That will install the locked dependency versions.