public-roadmap
public-roadmap copied to clipboard
checkly
Multi-step API check
Describe the solution you'd like For a CRUD API set, it makes sense to test these in a multi-step sequence. For example, to do a POST to create an object, assert that it was created in the response, then POST to delete the object and assert the response.
Describe alternatives you've considered Could potentially do this in setup/teardown scripts, although then it is less clear in the monitoring.
@zswanson thanks for adding this. For now, we have actually actively resisted the multi-step way because it creates some interesting issues. What if the create step fails? What if the delete step fails? We could do retries, but do we want that always?
With setup/teardown steps we make things a bit more atomic — there is still room for things getting out of sync — and we add the option to add logic to retries: you can check if a value was actually created before trying to delete it. You can check if a value isn't already there before trying to create it.
I hope you see our thinking on this issue. However, we are actively developing Checkly and we'll revisit "chaining" as a feature at some stage.
Ok thanks, I'll just work at putting this type of sequence into the setup and teardowns.
On Tue, Oct 29, 2019 at 11:33 AM Tim Nolet [email protected] wrote:
@zswanson https://github.com/zswanson thanks for adding this. For now, we have actually actively resisted the multi-step way because it creates some interesting issues. What if the create step fails? What if the delete step fails? We could do retries, but do we want that always?
With setup/teardown steps we make things a bit more atomic — there is still room for things getting out of sync — and we add the option to add logic to retries: you can check if a value was actually created before trying to delete it. You can check if a value isn't already there before trying to create it.
I hope you see our thinking on this issue. However, we are actively developing Checkly and we'll revisit "chaining" as a feature at some stage.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/checkly/product-roadmap/issues/46?email_source=notifications&email_token=AAJWY6QSEDLCWG6TT4ZUBMTQRBJUJA5CNFSM4JB2J4Q2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECQ6TTI#issuecomment-547482061, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJWY6VZKYU3FKMKHPEY3B3QRBJUJANCNFSM4JB2J4QQ .
Missing this feature is actually the main reason why other business units in our company cannot adopt Checkly - we have numerous products which require pure API flows for continuous testing. While some of our products simply do not have a UI to test flows, it's also important to note that pure API testing allows for a different level of availability checks. By running UI and pure API flows side by side, we get a comprehensive view of our infrastructure and products. The difficulty in implementing this type of feature is understood, however I think that the user base for this type of feature is easily underestimated.
Commenting here as an upvote for this feature. It will be really helpful.
I support this feature, too. Please also keep in mind, that there will also be the need to use part of the response (e.g. an invitation token) of a former test has to be used in another test down the line. So it would be a useful addition to be able to at least use env vars in the payload.
thx for all your efforts
I would love this feature too.
It is blocking us for leaving Runscope completely.
It is maybe an anti pattern to make API dependable but also sometime very useful.
I would like this feature as well. I know there are downsides to testing APIs as a "flow" but customers/consumers use APIs in this way so it would be a great feature to have. Its a shame that there is no established "PlayWright" for API testing as PlayWright gives you that functionality for UI testing.
Used this a lot in Runscope and really miss it in Checkly. Including the option to save variables during earlier steps and then use them as part of the requests later steps of the test chain. Pretty much a must have feature for the API monitoring use cases I would use Checkly for.
