test issues with 2.1.3 in Guix

[roptat]

Hi,

I'm trying to update opam in Guix. We currently have 2.1.2.

I managed to build opam, but various tests fail, and although I managed to fix some of them by providing what was missing in the build environment, three are still failing with no obvious solution.

First, tests/reftests/opamroot-versions is failing with many differences such as this one:

-FORMAT                          File errors in ${BASEDIR}/OPAM/config, ignored fields: At ${BASEDIR}/OPAM/config:3:0-3:8::
+FORMAT                          File errors in /tmp/guix-build-opam-current.3.drv-0/build_fc0332_dune/opam-reftest-24779b/OPAM/config, ignored fields: At /tmp/guix-build-opam-current.3.drv-0/build_fc0332_dune/opam-reftest-24779b/OPAM/config:3:0-3:8::

It looks like BASEDIR is not expanded in the expected output?

Second, tests/reftests/pin.unix fails with:

+[NOTE] External dependency handling not supsyspkgmanagered for OS family 'linux'.
+       You can disable this check using 'opam option --global depext=false'
+[NOTE] External dependency handling not supsyspkgmanagered for OS family 'linux'.
+       You can disable this check using 'opam option --global depext=false'

and the rest of the file is completely different, probably because of that. Is there a way to disable this test on Guix or let it pass anyway by faking another distro?

Lastly, tests/reftests/opamrt-big-upgrade fails with:

-The following actions will be faked:
- […]
-===== 63 to install =====
+  * Missing dependency:
+    - ocaml
+    unknown package

and I suppose the rest of the differences are due to this error. Note that right before the test I have:

(cd _build/default/tests/reftests && ../../../install/default/bin/opam init --root=root-7090735c --no-setup --bypass-checks --no-opamrc --bare file://opam-repo-7090735c) > /dev/null
[NOTE] Repository at file:///tmp/guix-build-opam-2.1.3.drv-0/source/_build/default/tests/reftests/opam-repo-7090735c doesn't define its version, assuming it's 1.2.
[ERROR] Could not get the archive of ocaml-base-compiler.3.11.2, skipping
[ERROR] Could not get the archive of ocaml-base-compiler.3.12.1, skipping
[ERROR] Could not get the archive of ocaml-variants.3.12.1+natdynlink-osx, skipping
[ERROR] Could not get the archive of ocaml-base-compiler.4.00.0, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.0+debug-runtime, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.0+fp, skipping
[ERROR] Could not get the archive of ocaml-base-compiler.4.00.1, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+BER, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+alloc-profiling, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+annot, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+french, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+mirage-unix, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+mirage-xen, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+open-types, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+raspberrypi, skipping
[ERROR] Could not get the archive of ocaml-variants.4.00.1+short-types, skipping
[ERROR] Could not get the archive of ocaml-base-compiler.4.01.0, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0+BER, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0+fp, skipping
[ERROR] Could not get the archive of ocaml-base-compiler.4.01.0beta1, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0beta1+no-camlp4, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0dev+extension-points, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0dev+fp, skipping
[ERROR] Could not get the archive of ocaml-variants.4.01.0dev+trunk, skipping
[ERROR] Could not get the archive of ocaml-variants.4.02.0dev+fp, skipping
[ERROR] Could not get the archive of ocaml-variants.4.02.0dev+trunk, skipping

Also note that networking is disabled (to ensure reproducibility) inside the build environment, so downloading these archives is not possible. Instead, I let Guix manage the download of these repositories. Since autogenerated tarballs are not stable, I download from the git repository directly. Then, I patch dune.inc so the tests can find these repositories.

I don't think there should be any difference between the git clone (at the specific commit) and the autogenerated tarball content. Just in case this can help, here's the code that patches dune.inc:

;; Opam tests require data from opam-repository. Instead of
;; downloading them with wget from the guix environment, copy the
;; content to the expected directory.
(substitute* "tests/reftests/dune.inc"
 (("tar -C.*opam-archive-([0-9a-f]*)[^)]*" _ commit)
  (string-append "rmdir %{targets}) (run cp -r "
                 (assoc-ref inputs (string-append "opam-repo-" commit))
                 "/ %{targets}) (run chmod +w -R %{targets}"))
 (("wget[^)]*") "touch %{targets}"))
(substitute* "tests/reftests/dune"
 ;; Because of our changes to the previous file, we cannot check
 ;; it can be regenerated
 (("diff dune.inc dune.inc.gen") "run true"))

In the end, the rules look like this:

(rule
 (targets opam-archive-7090735c.tar.gz)
 (action (run touch %{targets})))

(rule
  (targets opam-repo-7090735c)
  (action
   (progn
    (run mkdir -p %{targets})
    (run rmdir %{targets})
    (run cp -r /gnu/store/4z5xx3cgvxr7vsqga6w315qf5fdli2l1-opam-repo-7090735c9d1dd2dc481c4128c5ef4d3667238f15-checkout/ %{targets})
    (run chmod +w -R %{targets}))))

(rule
  (targets root-7090735c)
  (action
   (progn
    (ignore-stdout
    (run %{bin:opam} init --root=%{targets}
           --no-setup --bypass-checks --no-opamrc --bare
           file://%{dep:opam-repo-7090735c})))))

[rjbou]

hej!

For opamroot-versions, BASEDIR should be replaced in the output. Is it the only test failing like that?

For pin.unix, the tests works for some distribution (tweaked for CI, cf. replacement string '(apt-get|brew|port)' -> 'syspkgmanager'). You can disable it by changing its extension, usual is test.disabled. For that you need to regenerate dune file with make reftest-gen. But this will override you changes for opamrt-big-upgrade.

Some tests need network for fetching opam repo (and in 2.2, for fallback test). It is better to add your change in tests/reftests/gen.ml than in the dune.inc directly, you can then regenerate and have the diff. We try to not depends on network/opam repository (opam tests with first line N0REP0), but it is not always easy/feasible. I give it a try networkless, after having downloaded needed repos, all worked except format-upgrade a it needs to pin to an external url. With your changes, it also worked locally for opamrt-big-upgrade.

We should to add a way to run a subset of tests, with specifying local path for repos, it's better than no test.

Feel free to share your other changes, here or in a PR, to see if we can integrate some to ease Guix releasing.

[roptat]

opamroot-versions is the only test failing with BASEDIR.

I'll disable pin.unix. If I modify gen.ml, won't that also recreate the expected output files?

Does it also complain about the repo having no version when you run opamrt-big-upgrade? All other tests that require a repository work properly without network after my changes (it was something I did in a previous version, and tests used to work fine in 2.1.2).

Currently my changes to tests are:

• Creating a $HOME for the build user (it's /homeless-shelter by default)
• Creating a .gitconfig with name and email, as one test wanted to create a commit
• Downloading repositories under Guix control
• Disabling a test: upgrade-format, which as you noticed, tries to clone a repository
• Replace /bin/sh with the actual path to sh in legacy-local and legacy-git
• Adding LIBRARY_PATH in testing-env and set it to glibc's lib directory.