-no-plugin semantics

[samoht]

I am curious about the intended semantics of -no-plugin. Currently -no-plugin seems to imply "discard any pre-compiled _build/myocamlbuild binary". Is that what it was intended?

In some cases it is useful to separate the build of _build/myocamlbuild and the actual run of that command (for instance building + generating the doc without having to remember the full list of build plugins, see dbuenzli/topkg#80). How are we supposed to do that with -no-plugin and such?

Thanks!

[gasche]

The default behavior of ocamlbuild is to, in a single run, build the plugin (from the plain ocamlbuild), then run it to handle the rest of the build. (If the plugin is already built, the first step is skipped.)

The specification of -no-plugin is that it will just run plain ocamlbuild, as if no myocamlbuild.ml existed -- it will apply the default rules and nothing else. I'm not sure whether you found a difference with respect to this specification.

If you want to build the plugin without running it on any targets, you can use the -just-plugin option. A second run of ocamlbuild without this option will directly run the built plugin, not rebuild it -- this is equivalent with running the plugin binary with the -no-plugin option.

Finally, if you want to build the plugin and run it at once, but you wish to pass arguments to the plugin run (and not the plugin build), you can use the (rather awkwardly named) -plugin-option (it's an option passed to the plugin run, not to the plugin build like -plugin-tag).

[gasche]

Note: the semantics of -no-plugin, -just-plugin, -plugin-option and -plugin-tags are explained in the manual, section: How myocamlbuild.ml is used. @samoht, @whitequark, do you see any clarification that I should make to this documentation?

[whitequark]

@gasche Yes. This passage:

The -plugin-option FOO option will pass the command-line option FOO to the myocamlbuild invocation — and ignore it during plugin compilation.

makes it not clear what happens to FOO if there's no myocamlbuild.ml.

[gasche]

What is the behavior that you would expect? Independently of what it does: currently there is a warning if -plugin-tags is used but there is no plugin (because it's a common pitfall for people to assume that -plugin-tag cppo.ocamlbuild doesn't require any myocamlbuild.ml on their side), we could do the same for -plugin-option.

[whitequark]

@gasche I would like the argument to -plugin-option to be used as if it was passed directly to ocamlbuild. Anything else would significantly complicate topkg (and oasis, and cross-compilation in general).

[gasche]

See #130's comment where I explain why there is no warning on the (apprently counter-intuitive) use of -just-plugin without a plugin. Do your tooling considerations imply that there similarly should be no warning in this case?

[whitequark]

@gasche Yes, absolutely.

[gasche]

So when there is no myocamlbuild.ml the current implementation does not perform two runs, it builds the target directly during the first run (the one that would usually just build the plugin and start it), as if -no-plugin was passed. This means that -plugin-option FOO is ignored altogether, which is not the behavior you want.

To fix this we need to either:

• move to an always-two-runs implementation (if there is no myocamlbuild.ml we just run ocamlbuild again?), which has a slight performance cost, but is more regular, or
• keep the "optimized" behavior but relax the conditions which switch to the two-run behavior: make a second run when either there is a plugin, or plugin-expecting options (-plugin-option and -plugin-tags) are passed

The first option sounds better long-term (simpler model, simpler code), but the second option would result in a less invasive patch whose user-observable effect-change would be easier to reason about.

The semantics of -plugin-option where there is no myocamlbuild.ml is clear. The semantics of -plugin-tags is less clear. If we just run system's ocamlbuild on the second run, they would be just ignored. Another option would be to, instead, always build a fresh _build/myocamlbuild.byte executable by linking ocamlbuildlib.cma and ocamlbuild.cmo, and calling ocamlfind and passing -plugin-tags if relevant. In effect, this would behave exactly as if we had an empty myocamlbuild.ml module. Pros and cons:

• on the plus side, this gives a very regular semantics that is easy to reason about; and it may be useful for some -plugin-tag choices (for example one could force-link a module that does some stuff such as installing an at_exit handler that reports run statistics; in effect that gives a way to introduce plugin-like behaviors without writing a myocamlbuild.ml file)
• on the minus side, this gives a more complex behavior (building a new plugin is more complex than calling the system's ocamlbuild directly) that is potentially more fragile

[whitequark]

To fix this we need to either:

what about third option, do the first run as if -plugin-option FOO was specified as FOO. surely it cannot be that hard?

If that's too hard then I'd go for "optimized" version; people tend to complain when a relatively niche behavior like this is forced onto them always, even if in real terms it does not amount to any significant slowdown.

[gasche]

The difficulty is that you can only know if there is a plugin or not after parsing the options, and that just parsing those FOO after-the-fact would break ordering expectations (eg. for -cflags). I guess it should be possible to clean the option state back to defaults, and reparse the whole thing in this case. (Note: in the general case cleaning and reparsing would be wrong as plugin code could set the options programmatically, and be linked before option parsing, so one would have to snapshot before parsing and go back to that state. But if we know that we are running stock ocamlbuild and not running from a plugin, we can avoid this logic.) (Note: in any case an API with configuration values instead of a global mutable configuration state would be a nice improvement to OCamlbuild's design and could help make it easier to configure and extend. But there are many other pieces of global state in ocamlbuild that would also need to change.)

On the other hand I like the idea that passing -plugin-tags provokes recompilation of an empty plugin (I think it may give more expressivity to experiment with plugins) so in that case I would like to go the "relax the conditions" route.

[whitequark]

I would like to go the "relax the conditions" route.

I agree that this seems a straightforward, and preferable, choice in light of your observations above.