Request body is not consistent across multiple runs ( no portman config changes made across runs )

[yajiv789]

I have found multiple issues in case of only this particular spec. [ Attached ] Observation-1 Request body is not consistent across multiple runs even if there is no change in portman configuration. Observation-2 Assignvariables is not working as expected. Instead of the assignVariable the path variables are replaced by random text. Observation-3 Inside the requestbody for one of the key which is an array, the items are duplicated. In the spec the example contains one item whereas in the postman collection I noticed two items. portman-issue.zip

[thim81]

hi @yajiv789

To be able to transform the OpenAPI specification into a Postman collection, it uses some logic to create requests with useable data.

This is the logical cascade on how request values are used/rendered:

1. If the request body/path/query property has an example sample, this will be taken
2. If no example sample, the CLI checks if the request body/path/query property has an example value defined, this will be taken
3. If no example sample or example value, the CLI checks if the request body/path/query property has a default value defined, this will be taken
4. If the request body/path/query properties has none of the above defined, it will render a random value based on the openapi-to-postman options

So if the conversion does not find the examples/default values, it will render a random value that will be different each time. To make it consistent across runs, make sure that each property at least has an example value defined OR overwrite them by using the Overwrite options in Portman.

Under the hood it is using: https://github.com/postmanlabs/openapi-to-postman/blob/develop/OPTIONS.md

• schemaFaker(default true): Whether or not schemas should be faked. If you set this to "false" no random values will be generated
• requestParametersResolution(default Example): Select whether to generate the response parameters based on the schema or the example in the schema. So by default, Portman will try to use the "Example" values defined; if not then the "SchemaFaker" setting will decide what to do.
• exampleParametersResolution (default Example): Select whether to generate the response parameters based on the schema or the example in the schema. So by default, Portman will try to use the "Example" values defined; if not then the "SchemaFaker" setting will decide what to do.

For more information on what these Postman settings do, you can browse the documentation of https://github.com/postmanlabs/openapi-to-postman

You can overwrite the openapi-to-postman options in Portman, like explained in the cli-options example

[thim81]

On observation 3: This is done by the default "schemaFaker" behaviour like explained here: https://github.com/postmanlabs/openapi-to-postman/issues/367#issuecomment-838227884

It is not ideal, but this can be overcome by defining a request body example sample OR by overwriting the whole array using Portman overwrite options.

[thim81]

on observation 2: The assign variables functionality is to set variables in Postman based on the Request responses, so that the request output can be re-used in subsequent requests. It does not set the variables upfront, for this you can leverage over overwriteRequestPathVariables or environment-variables-as-postman-variables

[yajiv789]

@thim81 Firstly Thanks alot for the detail explanation. I have gone through the above info provided by you, I have created the configurations as per the portman documentation. For rest of my specs it is working as expected but in the case of this specific spec I am facing this issue. It seems like 'allOf' is not handled, but I am not sure though. Following is the snippet from the full spec. components: schemas: CreatePlan: allOf: - $ref: "#/components/schemas/PlanContent" description: A request to create a pricing plan. required: - name - type - visibility

I have also attached the spec and portman configuration in the first comment. You can execute them to reproduce the observations.

[yajiv789]

@thim81 AssignVariables issue is a typo, Apologise its my bad. Hence observation-2 is invalid. But the other two issues are correct.

[thim81]

@yajiv789 We have made some improvements on supporting allOff, anyOff since Portman v1.17.0

Could you review if using the latest version, solved your observations?

[trmaphi]

@thim81 it seems like allOf haven't been resolved too, https://github.com/postmanlabs/openapi-to-postman/issues/390. It's a bug in openapi-to-postman package