mojo icon indicating copy to clipboard operation
mojo copied to clipboard

Support Test2 in Test::Mojo

Open kraih opened this issue 5 years ago • 33 comments

We should support Test2 out of the box, so we can do stuff like this.

use Test2::V0;
use Test2::Mojo;

my $t = Test2::Mojo->new( ... );

$t->get_ok( '/foo' )
    ->status_is(200)
    ->header_is( 'Content-Type' => match qr{application/json} )
    ->json_is( hash {
        field foo => E;
        field bar => DNE;
        end;
    } );

done_testing;

kraih avatar Aug 21 '19 08:08 kraih

I'd be happy to send in a PR for this.

From your snippet, it looks like you'd prefer to have Test::Mojo detect which test backend is loaded, and use that one (as opposed to having a separate Test2::Mojo, or something similar). Does this sound about right?

jjatria avatar Aug 21 '19 09:08 jjatria

I don't think detection is a good idea. Test::Builder and Test2 based testing modules can coexist so both might be loaded. It would be an incompatible behavioral change if not explicitly requested by the user.

Grinnz avatar Aug 21 '19 15:08 Grinnz

Other options would be, a Test2::Mojo, or an import argument for Test::Mojo (probably easier).

Grinnz avatar Aug 21 '19 15:08 Grinnz

what about such approach https://github.com/mojolicious/mojo/compare/master...elcamlost:Test2?expand=1

elcamlost avatar Aug 21 '19 15:08 elcamlost

@elcamlost: that's very tied to Test2::V0, which is currently the recommended bundle for Test2, but will likely change. And although not the recommended way to do it, you can use Test2 without using that bundle.

The Test2::Mojo seemed to me the cleanest in my initial attempt, since it fits into the naming scheme of Test2-related modules, and meant that the Test::Mojo code could be left alone to avoid breaking anything.

jjatria avatar Aug 21 '19 16:08 jjatria

I do not recommend using detection. As was already stated it is possible, in fact likely, that many people will use both the Test2 specific tools and some tools still built off of Test::Builder.

I like the idea of having a Test2::Mojo that co-exists or replaces Test::Mojo. This puts it in the test authors hands to pick the one that makes the most sense for their current test, and nobody has to guess what is actually happening.

Testing should be obvious, and you should know what a test is doing when you read the source. Having the code 'guess' what to do adds overhead for test authors. Make it explicit and obvious.

exodist avatar Aug 21 '19 18:08 exodist

Having Test2::Mojo and Test::Mojo makes things obvious in code. But then we have a pretty big documentation problem. You need to find a format to explain all the behaviour changes in Test2::Mojo and fit a good explanation into the guides.

kraih avatar Aug 21 '19 19:08 kraih

The other benefit of a separate Test2::Mojo is that it would avoid loading Test::More entirely if the user chooses that, and I believe that means Test2 can work more cleanly without Test::Builder compatibility. But this would also mean not reusing any code from Test::Mojo.

Grinnz avatar Aug 21 '19 19:08 Grinnz

Correct. Test2 has some extra compatibility layers thrown in whenever Test::Builder is loaded. A pure Test2 process will be more efficient and slightly faster (measurements may vary).

exodist avatar Aug 21 '19 19:08 exodist

If Test2::Mojo was a full fork of Test::Mojo i doubt it would qualify for inclusion in the Mojolicious distribution. It could be released as a 3rd party module for now and later replace Test::Mojo completely in the next major release maybe.

kraih avatar Aug 21 '19 19:08 kraih

Maybe Test::Mojo can be refactored into two packages? One is coupled with Mojolicious and independent from Test framework and another, which loads Test::More and does other work?

elcamlost avatar Aug 21 '19 21:08 elcamlost

Small gist to illustrate idea from ^^^

elcamlost avatar Aug 22 '19 18:08 elcamlost

@elcamlost If Test2::Mojo is to be thought of as a sort-of Test2 tool (as in Test2::Tools...) then maybe we could change the use line from Test2::V0 to use Test2::Tools::Basic (for diag) and Test2::Tools::Compare (for is, like, etc).

Then again, I'm not sure which of those is more Test2-like (going with the bundle, or with the tools directly).

jjatria avatar Aug 27 '19 08:08 jjatria

Probable trick with Test::Builder::Level should be rewritten too. See Test2::Manual::Tooling::Nesting.

elcamlost avatar Aug 27 '19 09:08 elcamlost

@elcamlost If Test2::Mojo is to be thought of as a sort-of Test2 tool (as in Test2::Tools...) then maybe we could change the use line from Test2::V0 to use Test2::Tools::Basic (for diag) and Test2::Tools::Compare (for is, like, etc).

Then again, I'm not sure which of those is more Test2-like (going with the bundle, or with the tools directly).

Test2 is flexible, either way works and is "the Test2 way", and that is by design. Pick the way that works best for you. You can make your own bundle, or you can use an existing bundle.

exodist avatar Aug 27 '19 16:08 exodist

Created first naive fork of Test::Mojo based on Test2.

@exodist I'm confused by error handling. If code inside Test2::Mojo dies like here, Test2 warns about unreleased context foreach call of ctx. Test::More is much less verbose in this case.

Have I done something wrong? Or Is it intended behavior?

elcamlost avatar Aug 27 '19 17:08 elcamlost

Releasing the context is super important, an exception prevents that. Usually test2 can detect it and be less verbose, but sometimes the detection fails.

You probably want https://metacpan.org/pod/Test2::API#context_do(&;@) in these use cases which wraps stuff in an eval for you, releases the context, then re-throws any exceptions after context is clean. Alternatively you could yse Try::Tiny or write your own evals.

exodist avatar Aug 27 '19 17:08 exodist

I finished adopting Test::Mojo code to Test2. Here it is. Of course, it needs documentation too, but code is written and test. BTW, writing tests for tests is surprisingly awesome ))

@kraih @Grinnz So what's the final decision? Should I make a PR or release it as independent module? If independent module, maybe repository should be in mojolicious namespace?

elcamlost avatar Aug 28 '19 21:08 elcamlost

Core cannot currently depend on Test2::Suite, so an independent module is the best option. It does not need to be in the mojolicious organization, but please indicate in the documentation that it is a fork of Test::Mojo, with the same copyright and author information (and yours added), and respect any wishes @kraih has for the namespace.

Grinnz avatar Aug 28 '19 21:08 Grinnz

Yes, please use a different namespace than Test2::Mojo. We'll keep that for Mojolicious core in case a reasonable solution is found.

kraih avatar Aug 28 '19 21:08 kraih

I'm a little disappointed that Test2 is not ready for use in Mojolicious. But a 3rd party module on CPAN is better than nothing.

kraih avatar Aug 28 '19 21:08 kraih

BTW, writing tests for tests is surprisingly awesome ))

One of the best parts about Test2 (besides the other best parts).

jberger avatar Aug 28 '19 21:08 jberger

I'm a little disappointed that Test2 is not ready for use in Mojolicious. But a 3rd party module on CPAN is better than nothing.

I am not sure what you mean by this. What do I need to do to make Test2 "ready" for mojo?

Or is the problem that Test2::Suite is not a 'core' module? in which case that is an issue to take up with p5p, I have no objections to it going core, but I also have no intentions of even putting a toe into any argument/debate over it, those fights are simply not worth having as I am completely and totally burned out on any "political" or "religious" debates around what modules go where. People who are "purist" and/or "gatekeepers" are simply too loud and obnoxious and not worth the effort. Not to say p5p falls into that category, but out of fear of such debates I have not even tried to suggest Test2::Suite go core.

exodist avatar Aug 28 '19 21:08 exodist

Yes, it's that it's not a core distribution, and the Test2 components in Test-Simple are of course too low level to really be useful on their own. Hopefully p5p is moving towards having some process or defined stakeholders around what modules are core.

Grinnz avatar Aug 28 '19 21:08 Grinnz

Well, if anyone who is interested here wants to suggest it go core, please do, and have my blessing. I will also be willing to answer technical questions that may come up in the proposal process. I will not however engage in any non-technical discussion/debate/etc over the issue with anyone who does resist.

exodist avatar Aug 28 '19 21:08 exodist

Is Test2::MojoX a good name?

On 29/08/2019 00:20, Sebastian Riedel wrote:

Yes, please use a different namespace than |Test2::Mojo|. We'll keep that for Mojolicious core in case a reasonable solution is found.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mojolicious/mojo/issues/1399?email_source=notifications&email_token=ABHSX4OL3VPSDPEUJ2EYSOTQG3TZPA5CNFSM4IODY2NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5MQFMI#issuecomment-525927089, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHSX4LOULMJMYDPUKLLHCLQG3TZPANCNFSM4IODY2NA.

elcamlost avatar Aug 28 '19 21:08 elcamlost

If the goal is to keep the Test2::Mojo namespace for Mojo core, why not go ahead and call the thing Test2::Mojo, then when/if Test2::Suite goes into perl-core the Test2::Mojo package can just move into mojo-core? Basically merge it into the other distro... Just make sure the version numbers are compatible, and that Test2::Mojo never has a version higher than mojo-core, then merging is easy.

exodist avatar Aug 28 '19 21:08 exodist

that also makes it a seamless transition for anyone using it, they will not later have to go s/Test2::MojoX/Test2::Mojo/ everywhere.

exodist avatar Aug 28 '19 21:08 exodist

The idea is complicated by that Mojolicious is the only package in the distribution with a defined version, and that incompatible changes may be desired when moving it into core. I think for both reasons using a different name is simply easier.

Grinnz avatar Aug 28 '19 22:08 Grinnz

Actually Mojolicious code already depends from external modules (IO::Socket::SSL, Role::Tiny etc), but they marked as optional. Why not add Test2::Suite to this list?

elcamlost avatar Aug 29 '19 09:08 elcamlost