postman-contract-test-generator icon indicating copy to clipboard operation
postman-contract-test-generator copied to clipboard

Does it support permutations on optional query parameters?

Open jarlah opened this issue 1 year ago • 6 comments

I have a basic example working perfectly, but the problem is that this "basic example" is an endpoint with a lot of optional query parameters. Therefore I see test output like this:

GET - /my/query/param/endpoint - No Request Body - SUCCESS: true - Has expected status code
GET - /my/query/param/endpoint - No Request Body - SUCCESS: true - Status code (200) is allowed
GET - /my/query/param/endpoint - No Request Body - SUCCESS: true - Has expected response body schema

which is ok I guess, to just test that the endpoint works in general. but it does not test permutations on query params...

Is that something omitted on purpose @allenheltondev ? If needed I could probably whip something up if I get some pointers

jarlah avatar Jan 04 '24 11:01 jarlah

to be honest, it only seems to make test permutations for request body, which is not really helpful for path parameters and query params based endpoints?

jarlah avatar Jan 04 '24 12:01 jarlah

I have an endpoint with aprox 40 optional query parameters. The simplest way to add tests for these query params is to make 40 additional tests dynamically for the 40 query parameters. Since we have query param validation, any invalid query param will fail. Thus ensuring that all documented query params is allowed and returns 200 OK (and also work with the provided examples)

jarlah avatar Jan 04 '24 12:01 jarlah

I see now that the code is using query params if they are required. So I would guess it should be quite possible to "fix" this issue of mine :)

jarlah avatar Jan 04 '24 12:01 jarlah

I have now referenced the optional query params properly, so they are not inlined but referenced. It didn't help.

Screenshot 2024-01-04 at 14 11 28

jarlah avatar Jan 04 '24 13:01 jarlah

at least make one test with all optional query params and another with no optional query params. What do you think @allenheltondev ?

jarlah avatar Jan 04 '24 14:01 jarlah

dang .. you say right here that its designed to only mutate required request body params

The request body will be generated with the required fields only. The generator will take the request body with all required fields and create an array of mutations against it. For each required field in the request body, two mutations will be created:

Omit the required field
Leave the required field blank
Once all the mutations have been created, the generator proceeds to execute the tests.

Well well I hope both the implementation and the readme can be updated to support optional or default mutation of optional query params too. Path params is not optional so its only about query params.

jarlah avatar Jan 04 '24 15:01 jarlah