opam
opam copied to clipboard
ocaml
[2.1.0~alpha] opam show is noticably slower, loading depexts
Using opam master (https://github.com/ocaml/opam/commit/3def1a6d1ff102aaa608f5e9170783b96f55069d), the following command:
opam show ocamlfind
takes less than 0.3s on opam 2.0.7 vs. almost 2s with opam 2.1.0~alpha.
This can come from the depext check. At each switch load, as for external file dependencies, depext checks that required system packages are still present.
This check takes ~1s on my debian, but @AltGr fix in #4165, it is now ~0.2s. It result on the full command taking ~1s with my huge opam root, and ~0.5 with an empty one.
You can see with --debug timestamps what is the action that is slower in 2.1.0.
The depext check is lazy and I believe it shouldn't be forced on opam show :thinking:
I don't reproduce such a delay, so your --debug output would be helpful.
It is also possible that the repository cache was scrapped (it's the case if e.g. you just updated) ; and it was not yet rebuilt (that needs a write lock, so it's only done on write-enabled commands). Do you reproduce after a simple install ?
@rjbou found it: there is actually a Lazy.force used to get the latest version, only when you opam show a package that is not installed (in which case, the installed version is shown) ; this explains why we had trouble reproducing.
It would be possible to print the highest version instead, to avoid the cost, but might be annoying in some cases (like packages where versions are actually triggers, e.g. xxx+system)
Or, more involved, it would be possible to compute the availability specifically for the package without forcing the whole stuff.
It is indeed possible to compute availability only with subset of targeted packages, but a package availability depends on depexts, we still need to load it. A more optimised solution is to load depexts only for targeted packages also, it's still an exec call, but given the system package manager it can be faster.
It would be possible to print the highest version instead
it is already the case, if pkg is not ① in pinned ② in installed ③ in available, the default is the highest version of all packages.
Also opam list --installed --short --safe --color=never is slow since 2.0.7, I noticed it because emacs runs that at startup and it takes seconds before I can use it…
@vbmithr You can see what takes time with --debug timestamps (disabled with --safe). As it is 2.0.7, it is not a depext loading taking time, please open a new issue.
With #4165 (latest master) it takes around 1s instead of 2 for me. However it is still 3x slower than with 2.0.7. Here is the output with --debug:
00:00.006 GSTATE LOAD-GLOBAL-STATE @ /home/kit_ty_kate/.opam
00:00.006 RSTATE LOAD-REPOSITORY-STATE @ /home/kit_ty_kate/.opam
00:00.050 RSTATE Loaded /home/kit_ty_kate/.opam/repo/state.cache in 0.043s
00:00.328 RSTATE Cache found
00:00.328 STATE LOAD-SWITCH-STATE @ 4.11
00:00.606 STATE Inferred invariant: from base packages { ocaml-variants.4.11.0+trunk }, (roots { ocaml-variants.4.11.0+trunk }) => ["ocaml-variants" {= "4.11.0+trunk"}]
00:00.608 STATE Switch state loaded in 0.279s
00:00.616 FILTER ERR: value_bool: "osx"
00:00.668 SYSTEM mkdir /tmp/opam-3392825-dd682e
00:00.721 SYSTEM rmdir /tmp/opam-3392825-dd682e
00:00.864 STATE depexts loaded in 0.197s
[...]
opam show --debug --raw ocaml-freestanding 0.82s user 0.07s system 99% cpu 0.891 total
From the log, the repo cache takes 0.3s and the invariant to infer 0.3 more. Setting the invariant with opam switch set-invariant would reduce that time.
wouldn't it be interesting to set this invariant automatically when opam detects an upgrade to 2.1 from previous versions? Isn't that related to #4188?
It is the same mechanism involved, indeed. My previous comment is more a hint, until this is fixed/improved/next release.
Just a follow-up on this: there is no change whether or not the switch has been upgraded from 2.0 or created fresh.