emacs-buttercup icon indicating copy to clipboard operation
emacs-buttercup copied to clipboard

FR: add a tests-failed-hook to better show useful info on travis build failure

Open mikavilpas opened this issue 10 years ago • 2 comments

I'd like to improve buttercup by adding support for a user-defined hook that is run after any test has failed. This would make travis build outputs immensely more useful. Consider my scenario with omnisharp-emacs:

  • the tests start a subprocess (the omnisharp-roslyn server that serves code completions etc.)
  • the tests interact with this subprocess
  • if a test should fail, it's difficult to show the status of this subprocess

mikavilpas avatar Aug 21 '15 06:08 mikavilpas

I like the idea. Just to make sure this is the best option – would it not be a better idea to define your own matcher so that you can, for example, (expect process :to-be-in-my-desired-state) and that can emit the appropriate information?

jorgenschaefer avatar Aug 21 '15 07:08 jorgenschaefer

Defining my own matcher is an interesting idea. So far I've only used the existing ones and some that I ported over from the old test framework, that use cl-assert under the hood. I'll definitely think about it, thanks.

That could work, but actually in my tests I want to test the effects of the output of the subprocess. For example, it might instruct emacs to jump to a file in the solution, and the test then should verify that emacs has opened that file.

So while it could work, I think it's more intuitive to observe the effects.

mikavilpas avatar Aug 21 '15 07:08 mikavilpas