Ability to create set of tests for a specific response with a single click

[gerhardvr-finclude]

Is your feature request related to a problem? Please describe. While setting up tests is quite painless in Thunder Client, it still take a lot of time to set up tests for each key-value pair in a JSON response. This is especially true if the JSON response has a "number of levels of nesting".

Describe the solution you'd like I'd like to be able to simply click a button and have Thunder Client set up a simple, basic test for each key-value pair in the response.

For example, it should be possible for the below response to automatically have all the tests created.

{ "meta": { "response_code": 0, "requestId": "tkSsZB0y80RIZFZDgUNGDZGHJ" }, "voucher_detail": { "provider": "Utilipay", "external_reference": "", "utility_type": "W", "munic": "Demo Municipality", "SGC": "400268", "meter_number": "41250012634", "account_holder": "Test07", "account_number": "123457", "account_type": "3", "address": "", "location_ref": "", "can_vend": true, "min_vend": 0, "max_vend": 0, "denomination": "", "outstanding": 0 } }

Describe alternatives you've considered The only alternative that currently exist is to set up each test manually, which involves a lot of typing

Implementation:

[rangav]

Thanks @gerhardvr-finclude for the feedback, will add to roadmap.

[gerhardvr-finclude]

Thank you.

Reading my feature request, I'm thinking to myself "How lazy can you be?" :)

Cheers

[rangav]

No, actually its a good suggestion, I had something similar in my roadmap to auto-generate tests. Good that you requested for this now.

Had few questions in my mind about this?

• what tests user might be interested, like equal comparison or type checking etc..
• some fields might be auto-generated, i.e the value will always change for every request.
• should it generate for all fields in json or some fields
• let the auto generate create as many tests as possible and user delete those that are not required.

just throwing out some ideas to make it efficient or useful for others users.

let me know your feedback on this

[gerhardvr-finclude]

-->HI Ranga, I’ve been pondering the same questions you posed. The reality is that the permutations are adfinitum. Therefore I propose that a simple equal rule for each field be generated and be set equal to the value. In my experience this makes up 90% of the checks anyway. I’m in the habit of setting up mock requests and responses so that I can control the response. This makes it possible to tightly control the tests. (That reminds me of another feature request I need to make 😊 ) what tests user might be interested, like equal comparison or type checking etc..I think equal comparison is a good start. Maybe give the user an option to select what type of comparisons should be included by default? some fields might be auto-generated, i.e the value will always change for every request.Very true. However, typically even these fields need to be tested. For example, if it is an unique transaction ID, I may want to test whether it is a GUID, even if it changes every time. Then it still helps to have the simple equal comparison in place. It is very quick to change the comparison type and replace the value with a regex.should it generate for all fields in json or some fieldslet the auto generate create as many tests as possible and user deleteMy thinking is all fields. Good testing covers all fields. If the user do not want to test a particular field, then he/she can easily delete it.Now for my additional feature request. This may be a bit of a big ask though. This is something I can achieve in Postman, but even there it is not easy and does involve some damn nifty scripting. It also requires the ability to wrap a group of tests within an IF statement (more about that in a bit).All good testing should include positive as well as negative testing and if possible, multiple tests with different permutations. It more often than not means running the same test multiple times for the same endpoint, just changing some of the request parameters on every call. This can be done in Postman by leveraging the Pre-request script. I believe there is an easier way to achieve this that fits in quite well with the user interface approach you applied in Thunder Client. In the ideal world....I would like to be able to set up a list of call parameters for each request, that can be cycled through when doing a Run All on a collection. For example if I have the below request JSON{    "call_type": 18,    "supplier_entity_id": 102,    "calling_entity_id": 245,    "transaction_id": "sksfdfsddsfsfd",    "meter_number": "41250012634"}I would set up the below “parameter” list:{ “supplier_entity_id”: 102, “meter_number”: “41250012634” }{ “supplier_entity_id”: 102, “meter_number”: “99999999999” }{ “supplier_entity_id”: 236, “meter_number”: “4040404040” }{ “supplier_entity_id”: 236, “meter_number”: “4040404041” }{ “supplier_entity_id”: 236, “meter_number”: “4040404042” }{ “supplier_entity_id”: 84, “meter_number”: “1546726867145” } Using the above list, Thunder Client will then run the same test 6 times, each time replacing the value of the “supplier_entity_id” and the “meter_number” with the value of the next item in the “parameter” list. Now the reason for being able to group tests using basic IF rules. No 1 on my list above give me a “success” result and therefore I run all the tests to check that the response is valid as expected from a success result. No 2 on the other hand will give me a “error” result because in this case the meter number does not exist. More often than not, an error response will look different than a success response, therefore the tests will look different. Therefor, by being able to group tests and apply basic IF checks, only certain checks will be run in the case where an error response is received while other tests will be run when a success response is received.  I don’t know if I’m describing this very well. Most probably it is as clear as mud. I’m happy to create a small Postman collection with the tests included and send that if it will help. Again, I don’t think this is a small thing to do. Maybe some day, when you have nothing else left to do... 😊 All that being said, after writing all of this down I’m thinking that Thunder Client may not be the right tool for achieving this anyway. Not because I don’t think that it will be possible, but because I don’t believe that is the core purpose of Thunder Client. TC to my mind attempts to speed up testing of APIs without having to leave VS Code, and it does an exceptional job of this. The type of testing I describe should be done outside of VS Code altogether. Heck, even Postman is not really the right tool for it. It should be done with a tool whose core purpose is to automate the testing of APIS, including load testing, stress testing and all the other wonderful buzz words.  Best regards  Gerhard  From: Ranga VadhineniSent: Monday, 07 March 2022 18:35To: rangav/thunder-client-supportCc: gerhardvr-finclude; MentionSubject: Re: [rangav/thunder-client-support] Ability to create set of tests for a specific response with a single click (Issue #492) No, actually its a good suggestion, I had something similar in my roadmap to auto-generate tests. Good that you requested for this now.Had few questions in my mind about this?what tests user might be interested, like equal comparison or type checking etc..some fields might be auto-generated, i.e the value will always change for every request.should it generate for all fields in json or some fieldslet the auto generate create as many tests as possible and user delete those that are not required.just throwing out some ideas to make it efficient or useful for others users.let me know your feedback on this—Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.>   https://www.finclude.net | @.

[rangav]

@gerhardvr-finclude thanks for reply

your comment has formatting issue, may be you send via email.

[gerhardvr-finclude]

HI Ranga,

I’ve been pondering the same questions you posed. The reality is that the permutations are adfinitum. Therefore I propose that a simple equal rule for each field be generated and be set equal to the value. In my experience this makes up 90% of the checks anyway. I’m in the habit of setting up mock requests and responses so that I can control the response. This makes it possible to tightly control the tests. (That reminds me of another feature request I need to make 😊 )

1. what tests user might be interested, like equal comparison or type checking etc..

I think equal comparison is a good start. Maybe give the user an option to select what type of comparisons should be included by default? 2. some fields might be auto-generated, i.e the value will always change for every request.

Very true. However, typically even these fields need to be tested. For example, if it is an unique transaction ID, I may want to test whether it is a GUID, even if it changes every time. Then it still helps to have the simple equal comparison in place. It is very quick to change the comparison type and replace the value with a regex. 3. should it generate for all fields in json or some fields 4. let the auto generate create as many tests as possible and user delete

My thinking is all fields. Good testing covers all fields. If the user do not want to test a particular field, then he/she can easily delete it. Now for my additional feature request. This may be a bit of a big ask though. This is something I can achieve in Postman, but even there it is not easy and does involve some damn nifty scripting. It also requires the ability to wrap a group of tests within an IF statement (more about that in a bit). All good testing should include positive as well as negative testing and if possible, multiple tests with different permutations. It more often than not means running the same test multiple times for the same endpoint, just changing some of the request parameters on every call. This can be done in Postman by leveraging the Pre-request script. I believe there is an easier way to achieve this that fits in quite well with the user interface approach you applied in Thunder Client. In the ideal world.... I would like to be able to set up a list of call parameters for each request, that can be cycled through when doing a Run All on a collection. For example if I have the below request JSON { "call_type": 18, "supplier_entity_id": 102, "calling_entity_id": 245, "transaction_id": "sksfdfsddsfsfd", "meter_number": "41250012634" }

I would set up the below “parameter” list:

1. { “supplier_entity_id”: 102, “meter_number”: “41250012634” }
2. { “supplier_entity_id”: 102, “meter_number”: “99999999999” }
3. { “supplier_entity_id”: 236, “meter_number”: “4040404040” }
4. { “supplier_entity_id”: 236, “meter_number”: “4040404041” }
5. { “supplier_entity_id”: 236, “meter_number”: “4040404042” }
6. { “supplier_entity_id”: 84, “meter_number”: “1546726867145” }

Using the above list, Thunder Client will then run the same test 6 times, each time replacing the value of the “supplier_entity_id” and the “meter_number” with the value of the next item in the “parameter” list.

Now the reason for being able to group tests using basic IF rules. No 1 on my list above give me a “success” result and therefore I run all the tests to check that the response is valid as expected from a success result. No 2 on the other hand will give me a “error” result because in this case the meter number does not exist. More often than not, an error response will look different than a success response, therefore the tests will look different. Therefor, by being able to group tests and apply basic IF checks, only certain checks will be run in the case where an error response is received while other tests will be run when a success response is received.

I don’t know if I’m describing this very well. Most probably it is as clear as mud. I’m happy to create a small Postman collection with the tests included and send that if it will help.

Again, I don’t think this is a small thing to do. Maybe some day, when you have nothing else left to do... 😊 All that being said, after writing all of this down I’m thinking that Thunder Client may not be the right tool for achieving this anyway. Not because I don’t think that it will be possible, but because I don’t believe that is the core purpose of Thunder Client. TC to my mind attempts to speed up testing of APIs without having to leave VS Code, and it does an exceptional job of this. The type of testing I describe should be done outside of VS Code altogether. Heck, even Postman is not really the right tool for it. It should be done with a tool whose core purpose is to automate the testing of APIS, including load testing, stress testing and all the other wonderful buzz words.

Best regards

Gerhard

[rangav]

Thanks Gerhard, your feedback is noted