unite
unite copied to clipboard
eproxus
differences from eunit_formatters included in rebar3?
What are the differences between unite and eunit_formatters that rebar3 include https://github.com/seancribbs/eunit_formatters
Wondering if it would be a good idea to try consolidating into one project and using that in rebar3.
Would love to consolidate this. I think Sean and I started each project independently pretty much at the same time way back, if I remember correctly.
As far as I know, there are some major differences:
- diffs of assert results (example)
- handling of more weird EUnit edge cases
- more compact stack traces relative to the project root
- optional profiling
- MIT instead of Apache license
Ah, ok. I think all those differences are improvements. But will have to look closer to see if there are still things from eunit_formatters we would want.
By "optional profiling" do you mean it own't always print out the "10 slowest tests" output? That is a thing we were looking to do anyway.
Can color optionally be turned off?
And I wonder why eunit_formatter uses that binomial heap.. could be doing it by just keeping the list could have issues on really large test sets... I guess I should ask Sean that one :)
Found a difference. I have few projects with eunit tests but on pgo just now ended up with a crash that was handled a little better with eunit_formatters. With unite:
=ERROR REPORT==== 8-Jun-2018::16:27:45 ===
Error in process <0.360.0> with exit value:
{function_clause,
[{unite_compact,terminate,
[{error,
{error,function_clause,
[{unite_compact,handle_cancel,
[test,
[{id,[4,2,1,2]},
{reason,undefined},
{desc,undefined},
{source,{pgo_test,select_null_test_,0}},
{line,57}],
{s,-576460749959081868,
[[{id,[4,1,1,1]},
{status,
{error,
{error,
{try_clause,
[{'__info__',<0.395.0>,#Ref<0.4171044783.496500737.35754>,
{conn,<0.395.0>,#Port<0.126295>,default}}]},
[{pgo_pool,checkout_info,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,358}]},
{pgo_pool,checkout,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,51}]},
{pgo,query,3,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo.erl"},
{line,80}]},
{pgo_test,'-bad_query_parse_test_/0-fun-0-',0,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/test/pgo_test.erl"},
{line,41}]}]}}},
{desc,undefined},
{source,{pgo_test,bad_query_parse_test_,0}},
{line,41},
{time,9},
{output,[<<>>]}],
[{id,[4,2,1,1]},
{status,
{error,
{error,
{try_clause,
[{'__info__',<0.406.0>,#Ref<0.4171044783.496500737.35766>,
{conn,<0.406.0>,#Port<0.127175>,default}}]},
[{pgo_pool,checkout_info,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,358}]},
{pgo_pool,checkout,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,51}]},
{pgo,query,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo.erl"},
{line,63}]},
{pgo_test,'-select_null_test_/0-fun-0-',0,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/test/pgo_test.erl"},
{line,56}]}]}}},
{desc,undefined},
{source,{pgo_test,select_null_test_,0}},
{line,56},
{time,0},
{output,[<<>>]}]],
true,3}],
[{file,
"/home/tristan/Devel/rebar3/_build/default/lib/unite/src/unite_compact.erl"},
{line,57}]},
{eunit_listener,call,3,[{file,"eunit_listener.erl"},{line,137}]},
{eunit_listener,group_loop,4,[{file,"eunit_listener.erl"},{line,98}]},
{eunit_listener,'-init_fun/2-fun-0-',2,
[{file,"eunit_listener.erl"},{line,63}]}]}},
{s,-576460749959081868,
[[{id,[4,1,1,1]},
{status,
{error,
{error,
{try_clause,
[{'__info__',<0.395.0>,#Ref<0.4171044783.496500737.35754>,
{conn,<0.395.0>,#Port<0.126295>,default}}]},
[{pgo_pool,checkout_info,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,358}]},
{pgo_pool,checkout,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,51}]},
{pgo,query,3,
[{file,"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo.erl"},
{line,80}]},
{pgo_test,'-bad_query_parse_test_/0-fun-0-',0,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/test/pgo_test.erl"},
{line,41}]}]}}},
{desc,undefined},
{source,{pgo_test,bad_query_parse_test_,0}},
{line,41},
{time,9},
{output,[<<>>]}],
[{id,[4,2,1,1]},
{status,
{error,
{error,
{try_clause,
[{'__info__',<0.406.0>,#Ref<0.4171044783.496500737.35766>,
{conn,<0.406.0>,#Port<0.127175>,default}}]},
[{pgo_pool,checkout_info,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,358}]},
{pgo_pool,checkout,2,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo_pool.erl"},
{line,51}]},
{pgo,query,2,
[{file,"/home/tristan/Devel/pgo/_build/test/lib/pgo/src/pgo.erl"},
{line,63}]},
{pgo_test,'-select_null_test_/0-fun-0-',0,
[{file,
"/home/tristan/Devel/pgo/_build/test/lib/pgo/test/pgo_test.erl"},
{line,56}]}]}}},
{desc,undefined},
{source,{pgo_test,select_null_test_,0}},
{line,56},
{time,0},
{output,[<<>>]}]],
true,3}],
[{file,
"/home/tristan/Devel/rebar3/_build/default/lib/unite/src/unite_compact.erl"},
{line,66}]},
{eunit_listener,call,3,[{file,"eunit_listener.erl"},{line,137}]},
{eunit_listener,call,3,[{file,"eunit_listener.erl"},{line,143}]},
{eunit_listener,group_loop,4,[{file,"eunit_listener.erl"},{line,98}]},
{eunit_listener,'-init_fun/2-fun-0-',2,
[{file,"eunit_listener.erl"},{line,63}]}]}
With eunit formatters:
1) pgo_test:bad_query_parse_test_/0:41
{try_clause,[{'__info__',<0.388.0>,#Ref<0.3446291608.3717201921.149427>,
{conn,<0.388.0>,#Port<0.14559>,default}}]}
%% /home/tristan/Devel/pgo/_build/test/lib/pgo/test/pgo_test.erl:41:in `pgo_test:-bad_query_parse_test_/0-fun-0-/0`
Output:
Output:
Sure, Unite is only opportunisticly written and doesn't handle every case, since the EUnit API is not properly documented. I can certainly add that case, and I'd consider every case not handled a bug (a nicer default than a function clause error is probably also warranted).
Colors can't be turned off currently, but the feature could be added easily.
@tsloughter I made some improvements to the handling of cancelled test cases in db8ebdda9600bceb72eaf76e4a085b7f590c4695. I would love for this to progress, could you try it out with your code above and see if it solves the problem?
Finally playing with this again. I think my only concern now is that this adds a couple more dependencies, one being somewhat redundant (colored) when included into rebar3.
I need to see if @ferd has any thoughts on this too.
As an illustration of differences, and I think reason to switch:
Output from eunit_formatters:
..................................................F
Failures:
1) pgo_test:custom_enum_test_/0:1048
Failure/Error: ?assertMatch(# { command := select , rows := [ { { mood , << "sad" >> } } ] }, pgo : query ( "select 'sad'::mood;" , [ ] ))
expected: = # { command := select , rows := [ { { mood , << "sad" >> } } ] }
got: #{command => select,num_rows => 1,
rows => [{{16419,<<"sad">>}}]}
%% eunit_proc.erl:325:in `eunit_proc:with_timeout/3`
Output:
Output:
Finished in 0.520 seconds
51 tests, 1 failures
From unite:
..................................................F
1) custom_enum_test_/0 (_build/test/lib/pgo/test/pgo_test.erl:1056)
Assert match failed!
Expression:
pgo:query("select 'sad'::mood;", [])
Pattern:
#{command := select,rows := [{{mood,<<"sad">>}}]}
Actual:
#{command => select,num_rows => 1,rows => [{{16618,<<"sad">>}}]}
50 tests passed 1 test failed (0.64 s)
@eproxus so vagabond brought up Output on irc. Does unite capture io that it can then print out if a test fails? I guess eunit_formatters does, I know that @ferd has rebar3's common test supporting this as well.
I'm fine with the switch if eunit users end up having a better experience. I personally don't use the tool, so I otherwise don't have a stake in this aside from maintainership.
@tsloughter As far as I know, it's EUnit that captures output, Unite only pretty prints it. I did add functionality that only prints the output if there is any, that's why you don't see it normally.
Regarding color, I could implement the colors natively to reduce the number of dependencies. How does Rebar 3 do color? Also natively?
rebar3 uses https://github.com/project-fifo/cf -- best to just rely on that, it can also detect if colors are supported by the terminal, in many cases they aren't when people are running tests automatically.
Regarding output, ok. I guess I should also just try it really quick with some io:formats in a test to see a comparison between the two.
@tsloughter Unite relies on io:format/2 taking binary formats, which cf doesn't support. Also, cf seems to always output a reset code at the end of every printout even though no reset has been specified. 😕 I could replace color with cf, but it would be a much bigger refactoring than I initially thought...
Another reason cf is inferior is that it forces code to make another pass over the whole output string instead of just dumping color escape codes inline into IOData structures.
Can cf be patched to not have these bugs instead of having to support and maintain two libraries at once?
Yea, that would be best. We'll have to ask @Licenser if stuff like the reset on every printout is a purposeful feature.
Hey! It resets to prevent confusion (i.e. people forgetting to set a reset). It made sense to me that a print statement (and the colors specified) are done at the end of the statement.
I'm not against adding a flag or something to disable this but I'm curious what how that is used to better understand it.
binary formats: not a big deal can add that!
@Licenser The problem is that you can't set a format in one call, then prints lot's of text (in a loop for example) and then set/reset the format. In my opinion, people who forget to reset should not be catered to 😉
Regarding binary formats, I think Erlang 21 stopped allowing iodata() as a format (which used to be possible). So I'm fine with the library not being able to handle that 🙂 (binaries would be nice of course)