Inaccurate output of retried scenarios

[badeball]

👓 What did you see?

There's a difference in outputted JSON from cucumber-js compared to its messages piped through cucumber-json-formatter.

✅ What did you expect to see?

The JSON output of cucumber-js in case of a scenario passing on 2nd attempt.

[
  {
    "description": "",
    "elements": [
      {
        "description": "",
        "id": "a-feature;a-scenario",
        "keyword": "Scenario",
        "line": 2,
        "name": "a scenario",
        "steps": [
          {
            "arguments": [],
            "keyword": "Given ",
            "line": 3,
            "name": "a step",
            "match": {
              "location": "features/definitions.js:5"
            },
            "result": {
              "status": "passed",
              "duration": 0
            }
          }
        ],
        "tags": [],
        "type": "scenario"
      }
    ],
    "id": "a-feature",
    "line": 1,
    "keyword": "Feature",
    "name": "a feature",
    "tags": [],
    "uri": "features/foo.feature"
  }
]

However, when piping the messages output through cucumber-json-formatter I get the following.

[
  {
    "description": "",
    "elements": [
      {
        "description": "",
        "id": "a-feature;a-scenario",
        "keyword": "Scenario",
        "line": 2,
        "name": "a scenario",
        "steps": [
          {
            "keyword": "Given ",
            "line": 3,
            "name": "a step",
            "result": {
              "duration": 2000000,
              "status": "failed",
              "error_message": "Error: YOLO!\n    at World.\u003canonymous\u003e (/home/jonas/projects/json-report-examples/retried-test/features/definitions.js:9:11)"
            },
            "match": {
              "location": "features/definitions.js:5"
            }
          }
        ],
        "type": "scenario"
      },
      {
        "description": "",
        "id": "a-feature;a-scenario",
        "keyword": "Scenario",
        "line": 2,
        "name": "a scenario",
        "steps": [
          {
            "keyword": "Given ",
            "line": 3,
            "name": "a step",
            "result": {
              "status": "passed"
            },
            "match": {
              "location": "features/definitions.js:5"
            }
          }
        ],
        "type": "scenario"
      }
    ],
    "id": "a-feature",
    "keyword": "Feature",
    "line": 1,
    "name": "a feature",
    "uri": "features/foo.feature"
  }
]

Notice the diff (using json-diff).

 [
   {
-    tags: [
-    ]
     elements: [
       {
-        tags: [
-        ]
         steps: [
           {
-            arguments: [
-            ]
             result: {
+              error_message: "Error: YOLO!\n    at World.<anonymous> (/home/jonas/projects/json-report-examples/retried-test/features/definitions.js:9:11)"
-              status: "passed"
+              status: "failed"
-              duration: 0
+              duration: 2000000
             }
           }
         ]
       }
+      {
+        id: "a-feature;a-scenario"
+        keyword: "Scenario"
+        line: 2
+        name: "a scenario"
+        steps: [
+          {
+            keyword: "Given "
+            line: 3
+            name: "a step"
+            result: {
+              status: "passed"
+            }
+            match: {
+              location: "features/definitions.js:5"
+            }
+          }
+        ]
+        type: "scenario"
+      }
     ]
   }
 ]

📦 Which tool/library version are you using?

• cucumber-js: 8.1.2
• cucumber-json-formatter: 19.0.0

🔬 How could we reproduce it?

Steps to reproduce the behavior:

1. git clone https://github.com/badeball/reproducible-issues.git
2. cd reproducible-issues/cucumber/retried-scenario
3. npx @cucumber/cucumber
4. npx json-diff report.json <(cat report.ndjson | cucumber-json-formatter)