Unmarshalling newman results not working in postman-executor
Describe the bug
In https://github.com/kubeshop/testkube/blob/develop/contrib/executor/postman/pkg/runner/newman/metadata.go , Line 67++ :
The RunTimings struct declares ResponseAverage and ResponseSD as float64 but all other timings as int.
In my opinion they should be float64 also.
Line 1 of metadata.go says "struct autogenerated from output file" and i think this is the problem. When running newman without --verbose, some values are always 0 (or may be int) so unmarshalling them as int works. But if you run using ``--verbose'' timings have decimals.
Real world example:
"timings": {
"responseAverage": 931,
"responseMin": 931,
"responseMax": 931,
"responseSd": 0,
"dnsAverage": 245.9453359991312,
"dnsMin": 245.9453359991312,
"dnsMax": 245.9453359991312,
"dnsSd": 0,
"firstByteAverage": 224.44169799983501,
"firstByteMin": 224.44169799983501,
"firstByteMax": 224.44169799983501,
"firstByteSd": 0,
"started": 1691576601367,
"completed": 1691576602368
},
This causes a json: cannot unmarshal number 245.9453359991312 into Go struct field RunTimings.run.timings.dnsAverage of type int error in https://github.com/kubeshop/testkube/blob/develop/contrib/executor/postman/pkg/runner/newman/newman.go, line 126
To Reproduce Steps to reproduce the behavior:
- Run a collection with
--verbose(Note: output: see real live example above) - See error (Note: See error message in real live example above)
Expected behavior Unmarshalling working -> accept float values
Version / Cluster
- Which testkube version? -> latest
- What Kubernetes cluster? -> Azure AKS
- What Kubernetes version? -> 1.25.6
Screenshots
Additional context n/a
Hey, @RalfWenzel Can you please provide an example of postman collection for testing?
@RalfWenzel we need an example of this test to reproduce the issue to fix it. if you could provide please.
@TheBrunoLopes , hey, sorry, i was offline for a couple of weeks. Will prepare this now.