newman-reporter-htmlextra icon indicating copy to clipboard operation
newman-reporter-htmlextra copied to clipboard

[Bug] Wrong request tests count when using pm.sendRequest

Open Kriz1881 opened this issue 4 years ago • 4 comments

Describe The Problem When using pm.sendRequest, the test count in "Total Requests">request>"TEST INFORMATION" is doubled (same test is displayed with count 2).

The "TOTAL ASSERTIONS" and the "TOTAL FAILED TESTS" on the Summary page are displayed as expected, but the "Prerequest Scripts" and the "Test Scripts" have the wrong count.

Steps To Reproduce

  1. Add additional request to the (for example) Test tab like:
pm.sendRequest("https://postman-echo.com/get", function (err, response) {
    console.log(response.json());
});
  1. Run the collection
  2. Open the generated report
  3. Verify the "Prerequest Scripts" and the "Test Scripts" count
  4. Navigate to "Total Requests">someRequest>"TEST INFORMATION" and verify the number of test displayed

The command that you're using to run the collection, would be a real help or the newman script, if you're using one.

newman run C:\someFolder\someCollectionName.postman_collection.json --reporters cli,html,junit,htmlextra --reporter-htmlextra-export C:\someFolder\Report.html --reporter-htmlextra-logs --timeout-script 90000 --color off

Screenshots and Gifs image

image

Collection and Environment Files

Version and Platform Information:

  • newman Version 4.5.5
  • newman-reporter-htmlextra Package Version 1.17.1
  • Node Version 12.13.0
  • OS Windows 10

Additional Context Add any other context about the problem not mentioned, the more information that can be provided the better.

  • Are you using the reporter from the command line or with a script? A: Using the reporter from the command line

  • Are you using the default template or a custom version? A: Using the default template

  • If you are using a custom .hbs file, please include this on the report. A: No

  • Have you made any local changes to the index.js file? A: No

Kriz1881 avatar Jun 29 '20 14:06 Kriz1881

Hey @Kriz1881

Thank you for the detailed bug report. Unfortunately, this is an open issue with Newman rather than the downstream reporters.

It's been raised on this report a few times now:

  • https://github.com/DannyDainton/newman-reporter-htmlextra/issues/147
  • https://github.com/DannyDainton/newman-reporter-htmlextra/issues/161
  • https://github.com/DannyDainton/newman-reporter-htmlextra/issues/176

There's not really anything that I can do here as all the data, not just the sendRequest() data comes from Newman.

DannyDainton avatar Jun 29 '20 14:06 DannyDainton

Thank you @DannyDainton :)

Kriz1881 avatar Jun 29 '20 15:06 Kriz1881

I'm trying to see what I can do with the data that's coming through already but it would be a hack and I don't feel good about it 😬

DannyDainton avatar Jun 29 '20 16:06 DannyDainton

Looks like requests and prerequest-scripts variables are interchanged in Newman. I ran a collection with 5 requests where 1 request had 3 sendRequest inside pre-requests. newman summary shows pre-request=5 and request=8

deepakaggarwal7 avatar May 30 '22 14:05 deepakaggarwal7