Criterion
Criterion copied to clipboard
Snaipe
assert fail backtrace
In case an assert fails, now we got lines information from where the assert have been called. That line information is useful as long as the assert is called from an unique place, but when somebody wants to create a collection of asserts and bundle it together that code either should be bundled with a macro, or somehow the caller should pass in the callee line information.
That is at least sub-optimal, as personally I try to avoid macros or extra work. I was thinking it would be great if optionally the stack trace could be printed, when an assertion is failed.
If this is something you think would work, I would work on this.
This is definitely something that we would want. I wrote a while ago tizzy with the intent of eventually providing an API for Criterion to use, but got side-tracked since then. Maybe you could pick up that work as a basis?
Similarly, I think that it would be cool to also print some sort of stack trace when a test crashes, though the logistics behind this makes the implementation slightly more delicate:
Essentially, I'd like to avoid installing a signal handler in the test process, because users are themselves allowed to set signal handlers, and I don't really want to document "well you can install a signal handler for SIGSEGV/SIGBUS/whatever, but be sure to call our signal handler too because you'll be losing on precious stacktrace info". waitid does provide us a way to access the address that faulted in the worker, but at that point we don't have any access to the address space of the worker, because /proc/pid/mem and /proc/pid/maps apparently become empty when the process becomes a zombie.
We may have to rely on signal handler for that after all, but it would be interesting to toy with possibilities. For instance, I have no idea whether the worker's VM stays referenced if a process keeps an open file descriptor to /proc/pid/mem up to the point that the worker becomes a zombie.
Sure I'll check tizzy.
Yes, stracktrace in case of crash would be a nice feature. But I would split this, and deal with the crash stacktrace after the assert failure part is done.
Sure, that sounds reasonable. Having a strong foundation for backtrace should allow us to extend the feature with crash backtraces when we get to that point.
why not just use libunwind ?
some other options for crash backtraces would be ptrace() and core dumps. ptrace sounds much better than core dumps, which need global configuration, eventhough ptrace's hard to get right.
Last time I tried something remotely similar with libunwind (which one, by the way? I know there's at least two of them, since LLVM ships their own), I could not get it to compile and work reliably on all platforms criterion targeted. Maybe that has since changed.
ptrace is unfortunately not an option because it generally breaks debuggers. Not being able to debug your tests would suck pretty bad :(
Well if the process is being debugged and crashes, the user's debugger will be notified and you'll get a backtrace anyway. Otherwise, criterion's could do the job. I'm pretty sure you can detect whether a debugger is attached (either by just looking at criterion's arguments, or by noticing ptrace(PTRACE_TRACE_ME) failed)
Last time I tried gnu's libunwind seemingly worked pretty well, which does't actually mean much since I didn't take time to test it as extensively as you may have.
which one, by the way? I know there's at least two of them
Potentially both can be simultaneously supported via feature detection [example]. LLVM one has some limitations like lack of unw_get_save_loc and unw_get_accessors, when compared to HP one: https://github.com/libunwind/libunwind.
