newman icon indicating copy to clipboard operation
newman copied to clipboard

Failure Source Objects and Execution Objects only include last response (nested requests)

Open collin-rogers opened this issue 3 years ago • 0 comments

  1. Newman Version (can be found via newman -v): v5.3.2
  2. OS details (type, version, and architecture): macOS 12.5
  3. Are you using Newman as a library, or via the CLI? CLI
  4. Did you encounter this recently, or has this bug always been there: encountered recently
  5. Expected behaviour: When executing a collection that contains nested requests inside one request, all responses should be shown.
  6. Command / script used to run Newman: newman run $POSTMAN_API/collections/$POSTMAN_COLLECTION_UID?apikey=$POSTMAN_API_KEY -e $POSTMAN_API/environments/$ENVIRONMENT_ID?apikey=$POSTMAN_API_KEY --folder Setup --folder Update -r cli,json --verbose
  7. Sample collection, and auxiliary files (minus the sensitive details):
{
    "info": {
        "_postman_id": "6a223cda-e017-418b-a5fd-59fd1747e588",
        "name": "Example Collection",
        "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json",
        "_exporter_id": "19607531"
    },
    "item": [
        {
            "name": "ServiceFolder",
            "item": [
                {
                    "name": "ServiceEndpoint",
                    "item": [
                        {
                            "name": "Test endpoint with specific data",
                            "event": [
                                {
                                    "listen": "prerequest",
                                    "script": {
                                        "exec": [
                                            "pm.sendRequest({",
                                            "    url: `http://localhost:8080/api/endpoint`,",
                                            "    method: 'POST',",
                                            "    header: {",
                                            "        'Content-Type': 'application/vnd.api+json',",
                                            "        'authorization': `Bearer ${pm.environment.get('token')}`",
                                            "    },",
                                            "    body: {",
                                            "        mode: 'raw',",
                                            "        raw: JSON.stringify({",
                                            "            \"data\":\t{",
                                            "                \"type\": \"thing\",",
                                            "                \"attributes\": {",
                                            "                    \"thing-name\": \"before-update\",",
                                            "                    \"description\": \"Before update\"",
                                            "                } ",
                                            "            }",
                                            "        })",
                                            "    }",
                                            "}, (err, res) => {",
                                            "    pm.variables.set('updateID', res.json().data?.id || -2);",
                                            "    ",
                                            "})"
                                        ],
                                        "type": "text/javascript"
                                    }
                                },
                                {
                                    "listen": "test",
                                    "script": {
                                        "exec": [
                                            "pm.test('Res code should be 204', () => {",
                                            "    pm.expect(pm.response.code).to.eql(204)",
                                            "})",
                                            "",
                                            "pm.sendRequest({",
                                            "    url: `http://localhost:8080/api/endpoint/${pm.variables.get('updateID')}`,",
                                            "    method: 'GET',",
                                            "    header: {",
                                            "        'Content-Type': 'application/vnd.api+json',",
                                            "        'authorization': `Bearer ${pm.environment.get('token')}`",
                                            "    }",
                                            "}, (err, res) => {",
                                            "    pm.test('thing should have new thing name', () => {",
                                            "        pm.expect(res.json().data.attributes[\"thing-name\"]).to.eql('foo')",
                                            "    })",
                                            "})",
                                            "",
                                            "pm.sendRequest({",
                                            "    url: `http://localhost:8080/api/endpoint/${pm.variables.get('updateID')}`,",
                                            "    method: 'DELETE',",
                                            "    header: {",
                                            "        'Content-Type': 'application/vnd.api+json',",
                                            "        'authorization': `Bearer ${pm.environment.get('token')}`",
                                            "    }",
                                            "})"
                                        ],
                                        "type": "text/javascript"
                                    }
                                }
                            ],
                            "request": {
                                "method": "PATCH",
                                "header": [],
                                "body": {
                                    "mode": "raw",
                                    "raw": "{\n    \"data\":\t{\n        \"type\": \"thing\",\n        \"attributes\": {\n            \"thing-name\": \"foo\",\n            \"description\": \"This is my greatest description\"\n        } \n    }\n}",
                                    "options": {
                                        "raw": {
                                            "language": "json"
                                        }
                                    }
                                },
                                "url": {
                                    "raw": "http://localhost:8080/api/endpoint/:id",
                                    "protocol": "http",
                                    "host": ["localhost"],
                                    "port": "8080",
                                    "path": ["api", "endpoint", ":id"],
                                    "variable": [
                                        {
                                            "key": "id",
                                            "value": "1"
                                        }
                                    ]
                                }
                            },
                            "response": []
                        }
                    ]
                }
            ]
        }
    ]
}

This issue mostly effects my ability to create a custom reporter that includes the original request that the test is intended to target. I have other requests inside the test script to support assertions and facilitate cleanup, and the last request there is the one that is being shown in the data.summary object. Nowhere am I able to find the request information for the original request.

Steps to reproduce the problem:

  1. Create request with requests in pre-request scripts and test scripts
  2. Run collection via newman with debugger running
  3. Inspect data.summary.run.execution.response.

collin-rogers avatar Aug 03 '22 16:08 collin-rogers