how to hide steps in the report

[ctaepper]

trafficstars

I have a beforeEach, where the actor logs in

• open login page
• enter email address
• enter password
• click submit button

in the tests this is a ....attemptsTo(Start.LoggedInAs(email, password)) and results in a rather large first step in every test. is there a way to hide a complete step in the reports?

[nbarrett]

you could try creating this:

export class Login implements Task {

    static toMySite() {
        return new Login();
    }
    @step('{0} logs into the site')
    performAs(actor: PerformsTasks): PromiseLike<void> {
        return actor.attemptsTo(
            See.if(Navbar.LoginStatus, equals('Login')),
            Enter.theValue('[email protected]').into(LoginPage.userName),
            Enter.theValue('password').into(LoginPage.password),
            Click.on(LoginPage.loginButton),
            See.if(Navbar.LoginStatus, equals('Logout')),
        );
    }
}

then call it like this from your top level task:

    performAs(actor: PerformsTasks): PromiseLike<void> {
        return actor.attemptsTo(
           Login.toMySite()));

Then, when you render your html it will be wonderfully summarised/expandable to show the detail... welcome to the wonderful world of task decomposition! Oh, I forgot to mention that the @step annotation is important to add as that's the thing that causes the formatting to work...

[ctaepper]

thank you for your answer @nbarrett, unfortunately I wasn't clear enough. I am already doing it the way you described. However, even though this is only one mainstep, it does include a lot of substeps. I want to completely ignore the whole step for the reports. something like an additional decorator would be great. ignoring this step would also result in a little performance optimization, because the test would not need to take ~8 screenshots of all the substeps

export class Login implements Task {

    static toMySite() {
        return new Login();
    }
    @hidden
    @step('{0} logs into the site')
    performAs(actor: PerformsTasks): PromiseLike<void> {
        ...
    }
}

actually, what I would really want to do is to place this task a before, not beforeEach which could result in ignoring this task for the report according to @jan-molak proposal

[nbarrett]

Hi again @ctaepper - ah yes, I see what you are getting at now. Your requirement seems to be very much along the same lines as an issue I raised with the java serenity product in serenity-core #669. I think for full flexibility here, there needs to be a global way of turning instrumentation on/off, or even clearing previously generated instrumentation from the event bus, but I think we'd need the authors viewpoint on this! I'll be following this thread as I think your requirement is a valid one.

[ctaepper]

yes, reading your issue, thats exactly what I want

Oh, I forgot to mention that the @step annotation is important to add as that's the thing that causes the formatting to work...

semantically, omiting the @step decorator could lead to a hidden step. if its only job is to format the step in the report, this should be the way to go

[nbarrett]

Agreed, but I to clarify, my requirement (and I think maybe yours too?), is to use the same Interaction (presumably with the @step annotation added) and to control at run-time whether they produce instrumented output or not? Additionally, just because my Interaction doesn't contain an annotation, it might call library ones that do and currently these always produce instrumented output. That's why I think we need some kind of global on/off switch.

[ctaepper]

of course you are right, please disregard my brainfart 🤯

[jan-molak]

Interesting! I think in order to make this work the additional annotation would need to change the mode of the registered reporters...

OK, so here's a bit of a braindump.

The @step annotation decorates an Activity( so aTask or an Interaction) to make it emit two events:

• one when the activity is started
• another one when it's finished

The @hidden could emit at event before ActivityStarts to make the SerenityBDDReporter ignore any subsequent events and an event after ActivityFinished to put it back into its normal state.

I'm planning to work on the SerenityBDDReporter over the coming weeks, so I'll consider this requirement as well.

In the meantime - how should the @hidden annotation change the report? Should we report that a number of steps have been hidden, or just hide them and move along like nothing's happened? ;)

Thoughts?

[ctaepper]

my intention is to completely hide those steps. they are already "hidden" behind one single (first) step... reporting "hidden steps" would just change the wording

however, considering @nbarrett's comment. this should not be done directly in the task, because it doesnt seem to be the responsibility of the task to reveal or hide itsself in the report

it feels like it should rather be some kind of attemptsTo variation

performAs(actor: PerformsTasks): PromiseLike<void> {
  return actor.attemptsToSecretly(
    See.if(Navbar.LoginStatus, equals('Login')),
    Enter.theValue('[email protected]').into(LoginPage.userName),
    Enter.theValue('password').into(LoginPage.password),
    Click.on(LoginPage.loginButton),
    See.if(Navbar.LoginStatus, equals('Logout')),
   );
}

[jan-molak]

Interesting! Thanks for pointing that out.

It feels like attemptsToSecretly still makes it a responsibility of a task, doesn't it? Do you think it would be different from:

@hidden
performAs(actor: PerformsTasks): PromiseLike<void> {
  return actor.attemptsTo(
    See.if(Navbar.LoginStatus, equals('Login')),
    Enter.theValue('[email protected]').into(LoginPage.userName),
    Enter.theValue('password').into(LoginPage.password),
    Click.on(LoginPage.loginButton),
    See.if(Navbar.LoginStatus, equals('Logout')),
   );
}

[ctaepper]

the difference here that the @hidden decorator would always hide this task whenever I execute it. But I selectively want to hide the task, e.g. only if it is part of a beforeEach scenario setup. but if it is part of the actual test run, it should be reported.

[nbarrett]

I think it should be entirely a run-time not source-file choice as per comment earlier

[ctaepper]

just to make my case clear:

// tasks/login.ts
export class Login implements Task {
  public static withCredentials(email: string, password: string): Login {
    return new Login(email, password)
  }

  constructor(private email: string, private password: string) {}

  @step('{0} tries to login with credentials "#email / #password"')
  performAs(actor: PerformsTasks): PromiseLike<void> {
    return actor.attemptsTo(
      Enter.theValue(this.email).into(LoginForm.EmailAddress),
      Enter.theValue(this.password).into(LoginForm.Password),
      Click.on(LoginForm.Button)
    )
  }
}

// features/almost-any-feature.ts
...
beforeEach(() => stage
  .theActorCalled('User')
  .attemptsTo(Login.withCredentials('[email protected]', 'test124'))
...

even this is a little bit simplified... currently 90% of our e2e tests required a logged in user, so all step reports begin with the same step and do not only focus on the "task/test at hand"

so I would like to change the attemptsTo in beforeEach to be hidden. this has nothing to do with the task of logging in to be hidden itsself, I just want to hide this part of the steps in the current test report.

I do also have tests to report the login itsself. I would very much like to keep my Login Task and not have to duplicate that because one time it should be hidden and the other time it should be reported because the task is always the same

[ctaepper]

seeing this now. it could be something like theActorCalled('User').preparesHimselfBy(...) because semantically this is what he is doing. there could be a reporter setting to indicate wether the preparations should be included in the report

[jan-molak]

Hmm, not sure if "hide any prep steps" can be treated as a general rule, though. In many scenarios the prep step can be important. Also, how should we deal with the situations in which one of the nested tasks of our hidden task fails? We'd still need to report the entire tree to show the context, won't we? (Just thinking out loud here).

I agree with @nbarrett that this should be a runtime option. In which case something akin to Log4J configuration might work. For example:

# the format could be:
# path to task=Log level

tasks/login.ts/Login=All    # "All" means: log in setup (before/after) and scenarios

I'm not sure if that could work, what the log levels should be, and whether or not that could work for you? This sort of approach would require some proof-of-concept...

Thoughts?

[nbarrett]

I meant at runtime from a coding perspective in terms of any/all of these at the scenario level:

• actor.attemptsTo(clear.reporting())
• actor.attemptsTo(disable.reporting())
• actor.attemptsTo(enable.reporting())

[jan-molak]

Oh that's cool, thanks @nbarrett. This could work with that idea of events changing the behaviour of a reporter. So Disable/Enable reporting tasks could emit an event that affects the reporter. Is that what you had in mind?

[nbarrett]

Hi @jan-molak - not sure about the implementation...if your suggestion meets the reqiurement(s) we've mentioned then that'd be grand!

[auxmedrano]

this is exactly what I'm struggling, I need to hide a step in the report because it sends sensitive data in a request for authorization, hope there is an update, in the meantime I will try to do a workaround.

thanks you all for your efforts.

[jan-molak]

Hey guys, so I've been thinking about it some more.

I agree that showing sensitive information in the report is not a great idea, but don't think that hiding the entire step solves the problem either as it might negatively affect the debuggability of the test - in particular, what should happen if there's an error in the step we're hiding? Do we report it? Do we not report it? And so on.

So I wonder if the problem of hiding sensitive data could be solved by introducing a new question along those lines:

export class Redacted<A> extends Question<Promise<A>> {

    static valueOf<AT>(answerable: Answerable<AT>) {
        return new Redacted<AT>(answerable);
    }

    constructor(private readonly answerable: Answerable<A>) {
        super();
    }

    answeredBy(actor: AnswersQuestions & UsesAbilities): Promise<A> {
        return actor.answer(this.answerable);
    }

    toString() {
        return `[redacted]`;
    }
}

You'd use it to wrap any other question or static value, i.e

actorInTheSpotlight.attemptsTo(
    Enter.theValue(Redacted.valueOf(process.env.USERNAME)).into(LoginForm.username),
    // etc.
)

So while Redacted would pass the actual value to Enter, it would be reported as:

actor attempts to enter the value "[redacted]" into the username field

Thoughts?

[nbarrett]

Thanks @jan-molak for not giving up on this one and coming up with great suggestions as always. I have a couple of points on this:

• I think choosing to hide a step is a very different requirement to hiding sensitive information such as a password, so they should be treated differently. I like the idea of having a wrapper for this that can be applied to Enter Tasks, so that's a 👍 from me. However, I find the word redacted quite unusual and certainly have never popped it into any conversation I've had (or code I've written) :-) perhaps people would naturally choose something more straightforward like mask?
• So that leaves hiding steps programmatically which I am definitely a fan of and often choose to do this during setup tasks. I think the Serenity java product has solved this well, whereby a Task can Implement IsSilent for when the task and everything below it does not appear in reports and IsHidden for when just the current task is not reported on but nested tasks are. There's an article about this here - see section entitled "Silent Tasks" (IsHidden isn't discussed in the article as it was added later but can be found here). There is also a runtime configurable Interface CanBeSilent as well, where a boolean isSilent value can be passed to the constructer. So I think this covers all requirements really. The java product also shows steps that have failed regardless of their hidden state, so diagnostic information is not lost when you most need it.