Soft asserts

[cheq-spark]

Good morning, is there any support for soft asserts. I find myself having alot of test cases with multiple verification inside the test function and soft assert would be a really nice addition to this already great framwork.

[yashaka]

Hey!

Soft assertions is not something "classic" in automated testing.

In atomic, more unit style of tests it usually breaks the "atomicity".

In integration end to end style of tests - it signifies overdetalization/overengeneering. Such tests should usually not check so much of details.

If you do your tests right, in an optimal way - you don't need soft assertions.

Hence adding a feature to a library that will be used mainly as "workaround" or just bad design that motivate people to write not efficient tests - does not sound like a good choice:-)

Soft assertions are not hard to implement on your side. Most probably I will just provide an example of a code snippet here and this should be enough)

But let's see, I will think more on your proposal to add it to selene...

On Tue, Sep 1, 2020, 07:43 cheq-spark [email protected] wrote:

Good morning, is there any support for soft asserts. I find myself having alot of test cases with multiple verification inside the test function and soft assert would be a really nice addition to this already great framwork.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/yashaka/selene/issues/224, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO6ZHUHS65PGAWCGG5XQ7DSDR3XVANCNFSM4QRIE3HA .

[cheq-spark]

Thank you for your response. If you have the time, can you show me how to implement it on my end with selene. I'm currently working with your template here.

Hey! Soft assertions is not something "classic" in automated testing. In atomic, more unit style of tests it usually breaks the "atomicity". In integration end to end style of tests - it signifies overdetalization/overengeneering. Such tests should usually not check so much of details. If you do your tests right, in an optimal way - you don't need soft assertions. Hence adding a feature to a library that will be used mainly as "workaround" or just bad design that motivate people to write not efficient tests - does not sound like a good choice:-) Soft assertions are not hard to implement on your side. Most probably I will just provide an example of a code snippet here and this should be enough) But let's see, I will think more on your proposal to add it to selene... … On Tue, Sep 1, 2020, 07:43 cheq-spark @.***> wrote: Good morning, is there any support for soft asserts. I find myself having alot of test cases with multiple verification inside the test function and soft assert would be a really nice addition to this already great framwork. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#224>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO6ZHUHS65PGAWCGG5XQ7DSDR3XVANCNFSM4QRIE3HA .

[yashaka]

Another reason not to implement such things in a library like selene - is vague requirements. Soft assertions is something that can be interpreted in a lot of ways. When you implement "exact" API - everything is straightforward and objective. When you start to think on "softness" -  it start to be subjective - how much of softness would someone need? How to log them properly in the report? What to log and what to not log? When you have your project - it's easy to create some helpers for soft assertions that you can tune to your specific needs, log what you want in what format you want, add specific settings... But when you build a library for a everybody, it start to be much harder to find the "universal API" that will match everybody and that will be easy to maintain. That's why we try to build the API of selene to be flexible and powerfull being simple and versatile as much as possible...

Even now you can perform multiple assertions on an element in the following style:

browser.element('#foo').should(
    have.text('x').and_(
    have.value('y')).and_(
    have.attribute('data').value('z')))

Maybe we can make it cleaner in selene, allowing to accept conditions as varargs automatically combining them by "and":

browser.element('#foo').should(
    have.text('x'),
    have.value('y')),
    have.attribute('data').value('z'))

Maybe we can add one more condition, something like have.on_page:

foo = browser.element('#foo')
bars = browser.all('.bar')
browser.should(
    have.on_page(
        foo,
        have.text('x'),
        have.value('y')),
        have.attribute('data').value('z')),
    have.on_page(
        bars,
        have.size(10)))

And it also will have the "soft assertion" behavior. In the last example, you check 4 things:

• foo 1 have.text('x'), 2 have.value('y')), 3 have.attribute('data').value('z'))
• bars 4 have.size(10)

And if something failed - you will have only one failure with results of what passed/failed in one message.

I will think more on all this, and also provide some examples, probably will add them to the template you mentioned too;)