public-roadmap
public-roadmap copied to clipboard
checkly
Support JSON schema
Is your feature request related to a problem? Please describe. Our company uses "Design-first approach" regarding APIs so we always define OpenAPI spec before doing any coding. Because of that we are maximising benefits from having this spec, like validating data in servers, generating clients etc... You already have basic support for OpenAPI specs, but you only use routes. You don't use schema in any way.
Describe the solution you'd like When importing OpenAPI spec, not only generate test routes, but also generate JSON schema that will be used to do assertions without any user input.
Describe alternatives you've considered At least allow to define general data types for assertions :) So I could easily define "this field should be string with min-length of 2 characters" or "this should be floating point number" instead of giving exact value.
@ozonep Thanks for contributing this. I really like the idea! Over the last years I've seen scattered adoption of JSON schema, so that is why it has never made it into Checkly yet. I would love to also ask a question:
How do you see the life cycle of these specs? Importing once in quite easy, but once there is change we need to reflect that. Fields change, names change etc.
@ozonep How do you see the life cycle of these specs? Importing once in quite easy, but once there is change we need to reflect that. Fields change, names change etc.
You could do it the way Assertible does it - they have an option to provide permanent link to OpenAPI spec file. Whenever you change spec, you just click "Sync" so Assertion fetches these changes and updates tests accordingly.
And if you adopt JSON schema, another veeeery useful & cool feature will become possible :) Auto-generation of request data. Checkly will be able to generate random data based on schema to save users a lot of time + to check for edge cases. Example - if schema defines that "name" field should only be 2-10 characters long - let's send 1 character! Or 999 characters! Will it break API? And for some data that can't be auto-generated (like specific ids) user will be able to enter them manually.
Would be a killer feature.
Python project that has it: https://github.com/kiwicom/schemathesis
This is a major blocker for us and it's available in other platforms. Maybe can support with this lib? https://www.chaijs.com/plugins/chai-json-schema/
This is definitely needed and should be available by simply adding one of several well tested libraries in your runtime
