Add skipped tests in report table (CLI)

[amdouence]

1. Newman Version (can be found via newman -v): 5.0.0

2. OS details (type, version, and architecture): Microsoft Windows 10 Entreprise / 10.0.17134 N/A version 17134 / x64-based PC

3. Are you using Newman as a library, or via the CLI? CLI

4. Did you encounter this recently, or has this bug always been there: First time i use this. New feature proposal

5. Expected behaviour: Report number of skipped test at the end of execution

6. Command / script used to run Newman: newman run Test.postman_collection.json --color on

7. Sample collection, and auxiliary files (minus the sensitive details): Test.postman_collection.json.txt (remove the .txt file extension)

8. Screenshots (if applicable): Execution result of the attached collection

Test

→ Skipped test
  GET https://learning.postman.com/ [200 OK, 83.99KB, 264ms]
  √  Status code is success
  √  Body matches Postman
  1. Status is 500
  -  [skipped] Response time is less than 200ms

┌─────────────────────────┬────────────────────┬───────────────────┐
│                         │           executed │            failed │
├─────────────────────────┼────────────────────┼───────────────────┤
│              iterations │                  1 │                 0 │
├─────────────────────────┼────────────────────┼───────────────────┤
│                requests │                  1 │                 0 │
├─────────────────────────┼────────────────────┼───────────────────┤
│            test-scripts │                  1 │                 0 │
├─────────────────────────┼────────────────────┼───────────────────┤
│      prerequest-scripts │                  0 │                 0 │
├─────────────────────────┼────────────────────┼───────────────────┤
│              assertions │                  4 │                 1 │
├─────────────────────────┴────────────────────┴───────────────────┤
│ total run duration: 429ms                                        │
├──────────────────────────────────────────────────────────────────┤
│ total data received: 83.55KB (approx)                            │
├──────────────────────────────────────────────────────────────────┤
│ average response time: 264ms [min: 264ms, max: 264ms, s.d.: 0µs] │
└──────────────────────────────────────────────────────────────────┘

There is one step, with 4 tests :

• 2 tests ok
• 1 test skipped
• 1 test ko

In the result table, we can see the 4 assertions and 1 error but the skipped test doesn't appear. I could be usefull to see the number of skipped test vs success test (nb_success != nb_assertion - nb_error)

[DannyDainton]

Hey @amdouence,

Is this something that you would like to pick up and submit a PR for, all contributions are welcome. 🚀

[aksharma53]

Hey @DannyDainton @shamasis @codenirvana @umeshp7,

I have tried to integrate skipped test with new format. I want to ask is there any thing to change so that I add or remove before moving ahead .

[aritrasinha108]

Hii @DannyDainton @shamasis @codenirvana @umeshp7 If this issue is still open to contributions, I'll be happy to take this up.

[adityaofficial10]

@aritrasinha108 Bro, I'm working on it already.😅

[shamasis]

Before I get a chance to review the code, a quick glance at the very helpful screenshots tell me that the name of the row in table is not correct.

If we look and recognise the pattern, we can see that assertions is the count for test and it is at the bottom.

If I were to induce from there, I would add a row after assertions and call it skipped-assertions. Then put the count of skipped tests in second column under total and merge the next two cells of executed and passed. Then merge the two cells.

From the code perspective, the name of the key storing this value in summary is also something that has to be crafted for intuitive usage and along the same lines as above.

[aritrasinha108]

From the code perspective, the name of the key storing this value in summary is also something that has to be crafted for intuitive usage and along the same lines as above.

@shamasis, I wanted to ask if it's really required to store the skipped assertions in the summary as we already list out all the skipped assertions separately through a conditional statement. I mean, it's just a subset of a list of all the assertions. Can't we directly filter it to just display the no. of skipped assertions?

[aritrasinha108]

@shamasis Does this work? Please review and suggest any changes if required ?

[shamasis]

The screenshot looks good. Just need to not have 0 under the failed column since skipped tests cannot fail.

[shamasis]

From the code perspective, the name of the key storing this value in summary is also something that has to be crafted for intuitive usage and along the same lines as above.

@shamasis, I wanted to ask if it's really required to store the skipped assertions in the summary as we already list out all the skipped assertions separately through a conditional statement. I mean, it's just a subset of a list of all the assertions. Can't we directly filter it to just display the no. of skipped assertions?

We can. But it seems a right approach to read off of summary for this list. I will know more when you push PR.

[aritrasinha108]

We can. But it seems a right approach to read off of summary for this list. I will know more when you push PR. @shamasis I've pushed the PR, #2667, and stored it in summary as you had asked. I used the name skippedAssertions. Regarding the 0 in the failed column in skipped assertions, I tried to print it separately, and it looks like this. Does this work?? I haven't added the colors yet.

[shamasis]

Add an empty extra cell. Also, at this stage do reconcile with your GSOC mentor for final approvals.