Use system-num, if available

[dra27]

cf. https://github.com/ocaml/opam-repository/pull/11300#issuecomment-518722338

Given that num is stable, it should be possible to do the same trick in num as graphics - so in a system switch, if the system-installed version of num is the correct version (e.g. from homebrew's ocaml-num package), make installing num a no-op, otherwise build it and override the META in $SWITCHROOT/lib/num.

Happy to be assigned this, assuming no one has a strong objection to the principle of the change - I just haven't to time right now to amend the package and propose a PR.

[dbuenzli]

I would rather be against this. These tricks significantly hamper the ability for tools like odig or omod to correctly do their job as you'd need to make them aware of these install directories and how to look them up and bind a package name to them.

System compilers and their non opam managed ocamlc -where are already annoying enough, I constantly have to special case for them. In fact something I wanted to raise at some point was whether it would be feasible for system compilers to actually symlink the contents of ocaml -where in $(opam var lib)/ocaml. Note that it's the actual file contents that need to be symlinked as opam assumes it can write that directory.

Well in fact I could be for this if the same symlinking strategy is workable and applied to such a package. One problem I see though is how to keep the symlinks in sync with the system packages if those update.

One will tell me this is because you are not using META files, yes that's true but I thought there was a consensus at some point that the eco-system should get rid of that indirection and have something reasonable and simple to be upstreamed to do library lookup (the more complicated version being Leo's namespaces). The tools I mention actually make this a reality in the current eco-system.

[dra27]

@dbuenzli - even with the symlinking, don't odig/omod already/still have to cope with the files for num potentially being installed in two different places (either as a pure findlib package or in the OCaml stdlib dir)?

[dbuenzli]

The problem is that there is no convention at the moment. Things are not entirely settled and ad-hoc both in omod and odig, as I was waiting on Leo's namespace proposal but maybe we should try to move on regardless.

Also it seems upstream is not entirely cold about it but I think we can also move towards the model described below without too much upstream involvement and/or implementing new lookups in the compiler.

So it was not clear to me that system-num would install in ocamlc -where. Regarding what happens in odig/omod in that case:

1. I think Omod will find them if they are in ocamlc -where but if you want to express package constraints then the constraint names become different: ocaml/num (or ocaml.num) vs num depending on whether you are using a system switch or not; this is of course not desirable and due to the fact that Omod hardcodes the ocaml name to ocamlc -where.

2. Regarding odig that means the num in ocamlc -where will be attributed to the ocaml package but it won't be listed in the html docs. Too many opam managed packages were writing files into $(opam var lib)/ocaml which then became attributed to the ocaml package by odig which would pollute its documentation. For this reason I elected to only list the stdlib and compiler libs from the odig ocaml package page (upstream can override this listing by installing an index.mld file according to the odig conventions). This is likely not reasonable from odig's part.

The system I would like to be able to gradually converge to is that there is an OCAMLPATH where lookups are performed for names in directories from left to right. The first level in these directories will correspond to package names (AFAIU @lpw25's OCAMLNAMEPSACES is basically a more complicated version of that, that also allows binding names regardless of the directory names).

The idea is then that you could then simply set e.g. OCAMLPATH=$(ocamlc -where):$(opam var lib). Now tool like odig or omod have a simple lookup procedure for getting package names and attributing them files and/or constraining lookup procedures (e.g. for selecting variants ptime/jsoo vs ptime/os).

One catch though is that the stdlib is not segregated in a dir but written directly in ocamlc -where. I think we agree we would like to have the packages for ocaml be rooted in the ocaml name (ocaml/{stdlib,compiler-libs,threads}). Here are a few simple things that could be done but none is entirely satisfactory:

1. We ask system packagers to stop writing system package in ocamlc -where but rather to ocaml -where/site-lib or somewhere else and use that new location in OCAMLPATH. However this breaks users of -I +DIR they now need to write e.g. -I +site-lib/DIR. It also means that we still need to special case the ocaml package lookup which I personally would rather not.
2. The install of the stdlib and compiler libs is symlinked by the upstream install procedure in $(ocamlc -where)/ocaml/, so that ocaml -where can be used as a toplevel name for OCAMLPATH. I expect resistance from upstream on this.

I understand this is a bit of a brain dump. From a broader perspective that's my design goals from a naming perspective:

1. We want to be able to use the file system to determine (package) names and files that are associated to them. A simple convention for that is given an installation directory $LIBDIR to attribute the files in $LIBDIR/PKG to the package name PKG (and those of $LIBDIR/PKG/SUB to names PKG/SUB or PKG.SUB).
2. We want to support multiple package install sources, that is lookup in more than one $LIBDIR, hence have a notion of left-leaning package PATH for lookup (dubbed OCAMLPATH above).
3. We want to have as little impact on the existing world®.
4. In the specific case of opam we want to be able to refer to the package names regardless of whether we are dealing with a system switch or not. The problem we have now being given the current structure we want to treat both ocamlc -where as a toplevel $LIBDIR for OCAMLPATH and as the directory bound to the ocaml name.

[dbuenzli]

I think it's time to move on a bit on this issue upstream I tried to make a point for agreeing on an OCAMLPATH variable here.

[Qusonann]

Is there any way to force installation of the num package? I removed brew-ed ocaml-num:

brew uninstall --ignore-dependencies ocaml-num

But the problem still occurs:

# [ERROR] It appears that the num library was previously installed to your system
#         compiler's lib directory, probably by a faulty opam package.
#         You will need to remove arith_flags.*, arith_status.*, big_int.*,
#         int_misc.*, nat.*, num.*, ratio.*, nums.*, libnums.* and
#         stublibs/dllnums.* from /usr/local/lib/ocaml.

No files mentioned in the error message exist, yet installation fails.

---

UPD: init-ed opam from scratch (rm -rf ~/.opam) after removing brew-ed ocaml-num, installs ok now.

[utkarsh2102]

This same problem (same as @Qusonann's) hit me. It drove me crazy. After rm -rfing the ~/.opam fixed stuff for me.

Any ideal solution of dealing with this?

[XVilka]

@Qusonann @utkarsh2102 the best way to do that is:

ocamlfind remove num
ocamlfind remove num-top

P.S. That num problem happens from time to time on different systems in different times. Quite annoying always, since you already can forget how did you solve it last time.

[utkarsh2102]

Aaah, I see, thanks! 😀

[shonfeder]

I hit this regularly, so just bumping this to prevent automatic closure.

[edit: Feb, 2021] Back again after 2 months. lol, I wish this happened frequently enough that I could remember what I need to do to fix it or even where to come to find out :rofl: I'll see you next quarter, past me! :smiling_imp:

[edit2: Oct 3, 2021] Hello again, my old friend. Progress: I've remembered to search for this issue! In case it brings me hear quicker in the future, The specific error I sought was The installation of num failed at "make findlib-install".

[utkarsh2102]

Preventing the auto-close. Lalala.

[shonfeder]

As a stop-gap, anyone have an idea how difficult it would be to add a message to this error, along these lines:

This is a known bug with `num` (see https://github.com/ocaml/opam-repository/issues/14646)

To fix, run the following two commands, then try to reinstall the target package again:

ocamlfind remove num
ocamlfind remove num-top

If such a workaround would be welcome, and if I could get a pointer, I'd give a shot at adding it.

[dbuenzli]

I could get a pointer,

See here for post-failure messaging examples.

[shonfeder]

Oh nice! Just a matter of configuration change. Thanks :)

[shonfeder]

Thanks for the reminder to followup on making a PR to improve the messaging here :) -- replying here to keep the issue open.