dune

dune copied to clipboard

It's quite common for new users to not know about how OCaml does not record stack traces by default. This results in a bad debugging user experience by default. Since we already de-optimize OCaml binaries in development (using -opaque), we could consider also turning stack traces on as well. Additionally, this would mean that dune runtest and dune exec would include stacktraces by default.

I'm in favour of activating backtraces by default, but if I understand well, this proposal would only activate them by default when the executable is lunched via Dune, but not otherwise. I fear this may end up confusing users when they launch the binary by hand. Is there an easy way to activate backtraces in the compiled program, irrespective of how it is launched?

We could generate a shell script that does the equivalent of OCAMLRUNPARAM=b exec ..., but that would be problematic on windows. A better workaround to turn on backtraces would be welcome, but I can't think of anything.

Link an extra module that does Printexc.record_backtrace true? That's a bit overreaching maybe.

That's tricky, I'm not sure I'd like Dune messing with my backtrace settings. For example, performance can be impacted.

We already impact your performance in the development profile to improve user experience.

Indeed differences between the dev and release profile have been one of the most common usability problems I've seen users facing in Dune, I have found for example quite a few packages in the wild that were compiled using the dev profile due to user confusion.

I think there is a substantial difference about Dune using different sets of compile flags, than Dune messing with the OCaml runtime. The latter is very much unexpected, I think.

I would not wan't dune to mess with the OCaml runtime without my explicit approval as developer.

IMHO, this change could merit some more wider user discussion, and thoughts about the usability implications.

I wonder what upstream would think of this change, cc @gasche ?

I agree that the feature itself is very desirable, for example, I have a PR for coq-lsp so it will re-run failed Coq commands that are "anomalies" with backtraces enabled so users have an easier time submitting a bug report to Coq.

But it seems to me that getting this kind of features would require some more close collaboration with the applications themselves, in the sense of this kind of debugging functionality could live in a library that apps could take care of that.

This was discussed in the last team meeting. The consensus was that the benefits outweigh the cons and that having the backtraces on is the better default, especially for newcomers. Unless somebody from the compiler team has strong objections to this (or a better proposal), this proposal is accepted.

How would one disable the behavior if needed?

Simplest way would be to set OCAMLRUNPARAM in the env stanza. We could also add a dedicated option if that's preferred.

This was discussed in the last team meeting. The consensus was that the benefits outweigh the cons and that having the backtraces on is the better default, especially for newcomers. Unless somebody from the compiler team has strong objections to this (or a better proposal), this proposal is accepted.

I'm sorry I couldn't participate in the discussion. I'm still a bit unconvinced about this tho.

While it is true that in dev mode we suffer a performance impact, we are supposed to gain it back in compile time, so there is a clear tradeoff here (and actually in Coq it is not clear what is more efficient for devs, as optimized Coq is quite faster, as we will see).

With this proposal, we are always paying the price even if our program doesn't crash.

I've measured the impact of OCAMLRUNPARAM="b" by default in Coq dev profile, and the impact is pretty significant, close to 20%. I'm unsure making every OCaml program that is run in dev mode 10-20% slower is a good strategy, as in particular you only need backtraces when your program crashes. At least for my workflows it is much more time efficient to re-run the failed program with debug options (as indeed, often -bt on is not enough to debug complex programs, I usually want to enable extra program-specific debug flags).

If newcomers have problems enabling backtraces, dune init could enable that by default in the template.

Here are some Coq numbers:

# Avg of two runs (deviation was very small), stable CPU clock

# stdlib build, release profile, no backtraces
Command being timed: "dune build -j 1 --cache=disable -p coq-core,coq-stdlib"
User time (seconds): 263.23
System time (seconds): 52.88
Percent of CPU this job got: 100%
Elapsed (wall clock) time (h:mm:ss or m:ss): 5:15.55

# stdlib build, release profile, OCAMLRUNPARAM="b"
Command being timed: "dune build -j 1 --cache=disable -p coq-core,coq-stdlib"
User time (seconds): 342.12
System time (seconds): 24.59
Percent of CPU this job got: 100%
Elapsed (wall clock) time (h:mm:ss or m:ss): 6:05.98

# stdlib build, dev profile, no backtraces
Command being timed: "dune build -j 1 --cache=disable coq-stdlib.install"
User time (seconds): 320.35
System time (seconds): 53.07
Percent of CPU this job got: 100%
Elapsed (wall clock) time (h:mm:ss or m:ss): 6:13.27

# stdlib build, dev profile, OCAMLRUNPARAM="b"
Command being timed: "dune build -j 1 --cache=disable coq-stdlib.install"
User time (seconds): 402.40
System time (seconds): 25.12
Percent of CPU this job got: 100%
Elapsed (wall clock) time (h:mm:ss or m:ss): 7:07.12

Obviously there are some cases where backtraces by default is better, and you're free to turn it on. Nothing stops you from adding Printexc.record_backtrace false into coq or even controlling it with your own setting. Any program that is so sensitive to this option should just set it explicitly. Or even better, use raise_notrace when possible to avoid generating backtraces regardless of any global setting when it knows they would impact performance.

As a counter example to coq, we always run dune with backtraces and the impact to performance is minimal.

Also, the idea of having users require dune init is questionable. It will not work for existing projects, and plenty of new users get started with existing rather than brand new projects.

Stepping back, we are trying to decide what is the better default here. For experienced users, I'm not willing to concede that having no backtraces is better, but at least it's arguable. For beginners, there's nothing that can replace good errors by default. Now if we also take into account that defaults impact beginners much more than advanced users - who are by and large already familiar with the available options, then I think the decision is fairly easy. We choose good backtraces first, just like every other programming language in existence. There's no need to think we're special here.

Indeed Coq may not be representative of the typical OCaml program here, the impact is way too much. Note that we need the backtraces when we want to debug so maybe raise_notrace is too much. Also, I'm not sure the particular case of Dune is relevant to the problems here. We can find programs that are not affected, but certainly other workloads are much impacted by b (scientific computing for example).

Still my main fear about expectations / documentation remains tho; I think most dune users are confused about the role of profiles, and they may not expect a sudden decrease in performance of their test-suites for example due to a Dune upgrade.

Just imagine what would have likely happened for Coq if I hadn't seen this issue: we would have made dev builds 20% slower once we would have gotten a newer Dune, and likely not realized about it for likely months.

We choose good backtraces first, just like every other programming language in existence. There's no need to think we're special here.

I can also get backtraces from gdb in OCaml native, not sure what the scope of "every programming language in existence" is, but certainly I can find many counterexamples. Anyways that's orthogonal to this proposal.

Stepping back, we are trying to decide what is the better default here.

I would expect this to be a decision of OCaml itself, but indeed, YMMV.