xcp
xcp copied to clipboard
xcp-ng
Moving to OCaml 4.10.x
If you are interested in supporting the toolstack implementation of XCP-NG/Citrix Hypervisor, here is something we from Citrix would be interested in: moving to OCaml 4.10.x.
- Moving to OCaml 4.10.x involves two steps: updating the ocaml.spec file to have the new compiler available in the RPM build system. The Fedora ocaml.spec could be a starting point. The spec file includes some patches some of which can be dropped because the 4.10.x includes fixes.
- Before embarking on the point above it would be a good idea to use OCaml 4.10.x to compile all toolstack components in https://github.com/xapi-project/xs-opam/ using the new compiler and to update libraries as needed. The CI for xs-opam is already configured to use OCaml 4.08 and 4.09 and something similar could be done for OCaml 4.10 to identify what libraries and code needs to be updated. I expect that several libraries will require updates and that it would be a good idea to do this on a separate branch as I don't expect these changes to be compatible with OCaml 4.08.1 that is currently in use.
Note that we're behind regarding upstream libraries available on the official repositories:
$ env OPAMNO=true opam upgrade | grep -E "↗|∗"
↗ upgrade conf-pkg-config 1.1 to 1.2
↗ upgrade astring 0.8.3 to 0.8.4
↗ upgrade dune 2.5.0 to 2.6.1
↗ upgrade conf-libssl 1 to 2
↗ upgrade sexplib0 v0.12.0 to v0.14.0
↗ upgrade result 1.4 to 1.5
↗ upgrade qcheck-core 0.9 to 0.13
↗ upgrade ppx_tools 6.0+4.08.0 to 6.2
↗ upgrade ounit2 2.2.2 to 2.2.3
↗ upgrade menhirSdk 20200211 to 20200624
↗ upgrade menhirLib 20200211 to 20200624
↗ upgrade macaddr 3.1.0 to 5.0.0
↗ upgrade jane-street-headers v0.12.0 to v0.14.0
↗ upgrade integers 0.3.0 to 0.4.0
↗ upgrade dune-private-libs 2.5.0 to 2.6.1
↗ upgrade dune-build-info 2.5.0 to 2.6.1
↗ upgrade cstruct 5.1.1 to 5.2.0
↗ upgrade alcotest 1.1.0 to 1.2.1
↗ upgrade ocaml-migrate-parsetree 1.7.1 to 1.7.3
↗ upgrade ounit 2.2.2 to 2.2.3
↗ upgrade menhir 20200211 to 20200624
↗ upgrade ctypes 0.16.0 to 0.17.1
↗ upgrade dune-configurator 2.5.0 to 2.6.1
↗ upgrade eqaf 0.6 to 0.7
↗ upgrade cstruct-unix 5.1.1 to 5.2.0
↗ upgrade ipaddr 3.1.0 to 5.0.0
↗ upgrade angstrom 0.12.1 to 0.14.1
↗ upgrade ppx_tools_versioned 5.3.0 to 5.4.0
↗ upgrade qcheck-ounit 0.9 to 0.13
↗ upgrade lwt 4.5.0 to 5.3.0
↗ upgrade base v0.12.2 to v0.14.0
↗ upgrade mirage-crypto 0.6.2 to 0.8.1
↗ upgrade bisect_ppx 1.4.1 to 2.4.1
↗ upgrade qcheck 0.9 to 0.13
↗ upgrade cstruct-lwt 5.1.1 to 5.2.0
↗ upgrade variantslib v0.12.0 to v0.14.0
↗ upgrade typerep v0.12.0 to v0.14.0
↗ upgrade stdio v0.12.0 to v0.14.0
↗ upgrade parsexp v0.12.0 to v0.14.0
↗ upgrade fieldslib v0.12.0 to v0.14.0
↗ upgrade rpclib 6.1.0 to 7.1.0
↗ upgrade mirage-crypto-rng 0.6.2 to 0.8.1
↗ upgrade alcotest-lwt 1.1.0 to 1.2.1
↗ upgrade ppxlib 0.8.1 to 0.13.0
↗ upgrade sexplib v0.12.0 to v0.14.0
↗ upgrade rpclib-lwt 6.1.0 to 7.1.0
∗ install ppx_fixed_literal v0.14.0 [required by ppx_jane]
∗ install ppx_cold v0.14.0 [required by ppx_assert, ppx_base]
↗ upgrade ppx_variants_conv v0.12.0 to v0.14.0
↗ upgrade ppx_typerep_conv v0.12.0 to v0.14.0
↗ upgrade ppx_stable v0.12.0 to v0.14.0
↗ upgrade ppx_sexp_conv v0.12.0 to v0.14.0
↗ upgrade ppx_pipebang v0.12.0 to v0.14.0
↗ upgrade ppx_optional v0.12.0 to v0.14.0
↗ upgrade ppx_optcomp v0.12.0 to v0.14.0
↗ upgrade ppx_let v0.12.0 to v0.14.0
↗ upgrade ppx_js_style v0.12.0 to v0.14.0
↗ upgrade ppx_here v0.12.0 to v0.14.0
↗ upgrade ppx_fields_conv v0.12.0 to v0.14.0
↗ upgrade ppx_enumerate v0.12.0 to v0.14.0
↗ upgrade ppx_deriving_rpc 6.1.0 to 7.1.0
↗ upgrade ppx_compare v0.12.0 to v0.14.0
↗ upgrade ppx_cstruct 5.1.1 to 5.2.0
∗ install ipaddr-sexp 5.0.0 [required by conduit, conduit-lwt-unix]
↗ upgrade ppx_custom_printf v0.12.1 to v0.14.0
↗ upgrade mirage-crypto-pk 0.6.2 to 0.8.1
↗ upgrade ppx_sexp_value v0.12.0 to v0.14.0
↗ upgrade ppx_sexp_message v0.12.0 to v0.14.0
↗ upgrade ppx_fail v0.12.0 to v0.14.0
↗ upgrade ppx_hash v0.12.0 to v0.14.0
↗ upgrade ppx_assert v0.12.0 to v0.14.0
↗ upgrade conduit 1.4.0 to 2.1.0
↗ upgrade bin_prot v0.12.0 to v0.14.0
↗ upgrade x509 0.11.0 to 0.11.2
↗ upgrade cohttp 2.3.0 to 2.5.4
↗ upgrade ppx_base v0.12.0 to v0.14.0
↗ upgrade jst-config v0.12.0 to v0.14.0
↗ upgrade conduit-lwt 1.4.0 to 2.1.0
↗ upgrade ppx_bin_prot v0.12.1 to v0.14.0
↗ upgrade cohttp-lwt 2.3.0 to 2.5.4
∗ install ppx_string v0.14.0 [required by ppx_jane]
↗ upgrade time_now v0.12.0 to v0.14.0
↗ upgrade conduit-lwt-unix 1.4.0 to 2.2.2
↗ upgrade ppx_module_timer v0.12.0 to v0.14.0
↗ upgrade ppx_inline_test v0.12.0 to v0.14.0
↗ upgrade cohttp-lwt-unix 2.3.0 to 2.5.4
↗ upgrade ppx_expect v0.12.0 to v0.14.0
↗ upgrade ppx_bench v0.12.0 to v0.14.0
↗ upgrade splittable_random v0.12.0 to v0.14.0
↗ upgrade base_quickcheck v0.12.1 to v0.14.0
↗ upgrade ppx_jane v0.12.0 to v0.14.0
↗ upgrade base_bigstring v0.12.0 to v0.14.0
↗ upgrade core_kernel v0.12.3 to v0.14.0
∗ install timezone v0.14.0 [required by core]
↗ upgrade protocol_version_header v0.12.0 to v0.14.0
↗ upgrade async_kernel v0.12.0 to v0.14.0
↗ upgrade core v0.12.4 to v0.14.0
↗ upgrade async_rpc_kernel v0.12.0 to v0.14.0
↗ upgrade async_extra v0.12.0 to v0.14.0
↗ upgrade textutils v0.12.0 to v0.14.0
↗ upgrade async_unix v0.12.0 to v0.14.0
↗ upgrade async v0.12.0 to v0.14.0
↗ upgrade rpclib-async 6.1.0 to 7.1.0
↗ upgrade conduit-async 1.4.0 to 2.1.0
↗ upgrade async_find v0.12.0 to v0.14.0
↗ upgrade cohttp-async 2.3.0 to 2.5.4
↗ upgrade async_inotify v0.12.0 to v0.14.0
===== ∗ 5 ↻ 133 ↗ 102 =====
Looking at their metadata it looks like they should be compatible with 4.10 as well, but it's not sure and some might need upgrading to be compatible with the new version.
First, thanks a lot for asking this, we are really excited here to provide/package "cutting edge" stuff into XCP-ng. Thanks to our testing repositories, we might be able to provide a "full real case" scenario of usage with latest XAPI code to impatient/curious users.
It also might be interesting for some users/customers having issues with memory leaks. Testing packages from the repo of the same name is trivial, and thanks to yum, in general, it's painless to rollback (we'll have to check that if it's true for XAPI).
Let's move forward :+1:
I have imported Fedora's ocaml into https://github.com/xcp-ng-rpms/ocaml/tree/8.2-ocaml-4.10 See the diff from previous: https://github.com/xcp-ng-rpms/ocaml/commit/496f3e90495c3aca5af46f82ea4610fef6474e01
There are several unknowns to me: CA patches to drop or keep? Handling of the differences between XS/XCP-ng's previous spec file and that new one. I've done a few adaptations that seemed obvious to me, but also put questions as comments and I don't know if the files being removed from the package are to be expected.
Built RPMs as well as build logs are available at https://koji.xcp-ng.org/taskinfo?taskID=16764
- The CA patches can be dropped because these issues are fixed in the compiler.
- I believe there are some RedHat specific hardening steps and macros (
redhat-rpm-config) that I would not use. - For now I think its more important to have a compiler but work on the libraries. The diff looks otherwise sensible to me.
I've opened a draft PR for the libraries work. I reached the point where the minimum amount of libraries have been updated (dozens, already) for compatibility with 4.10 while retaining compatibility with 4.08.
Now there are build failures to handle.
4.10.1 is being integrated into the ci of the ocaml libraries repository: https://github.com/xapi-project/xs-opam/pull/514
This makes it even, slower so I've changed how the Ci is run to reduce runtime: https://github.com/xapi-project/xs-opam/pull/515
After that I'll integrate 4.10.1 into the RPM build system and test it before we drop 4.08.1.
Because of the large amount of changes needed to adopt 4.10 in existing ocaml libraries, Sam and I don't think it's feasable to get a build working on top of xcp-ng 8.2.
That would be better served for a build based on the latest commits in the toolstack. The integration into the citrix hypervisor is looking good and unless there are unexpected issues it should be available in future releases.
I'd like to thank @stormi for his work on the opam repository and in the spec file for ocaml, it's was necessary to get this effort rolling.