Dredd Validator functionality in cli

[alekhrycaiko]

Is your feature request related to a problem? Please describe.

I've run into issues with my swagger documents, where my document is valid swagger yaml (according to a validator I'm using). But, when I run Dredd I've experienced cases where warnings, or errors are generated when validating the file. I find the inconsistency a bit troubling. I'd ideally like to keep my Dredd runs warning and error free; and would even prefer to not run my Dredd suite if my swagger file is in a bad state.

Describe the solution you'd like

I'd love for there to be validator functionality built into the existing dredd cli.

E.g.

dredd validate <yaml file path>

I think a solution like this could be helpful for saving users time. In my situation, if the validator came back with a 0, I could avoid standing up Docker images, and then running Dredd against them - which could be quite a nice time savings!

In addition, a cli tool could be useful as a configurable pre-commit hook, and would help to ensure that swagger files are maintained properly for Dredd to use.

Describe alternatives you've considered

I've tried running an external validator, but, I found the validator didn't line up with what Dredd expected. e.g. of validator I used is here. e.g. https://github.com/APIDevTools/swagger-cli

[kylef]

@albertjan we provide a CLI called Fury which can be found at https://github.com/apiaryio/api-elements.js/tree/master/packages/fury-cli#fury-cli. This uses the same parser as Dredd and should provide you with similiar results. You could use this in continuous integration etc.

I think there is one additional validation that Dredd will perform, which is to warn if there is a missing default/example value for a parameter. Although this may be filled in via a Dredd hook if it isn't in the API Description Document.

That said; I know our OpenAPI 2 parser isn't in the best shape and some of the warnings or errors it presents are unclear or do not help you know what the problem is. That's one of the key differences between our support for OpenAPI 3 (https://blog.apiary.io/OAS-3).

[alekhrycaiko]

Thanks @kylef

I'll give the fury parser a look. And yes, I've noticed the improvements to the reporting in OAPI-3 . It's been awhile since I've dove into developing using Dredd, and I'm happy to see the improvements :)

Concerning the validation checking, I still think it may make sense to consider a validate feature. It'd be very ideal to have both the example/default cases validated, alongside what's covered on the fury-parser. Perhaps the Dredd Cli is the place to do that? Is there any interest in something like this being worked on?

[kylef]

Perhaps the --dry-run option of Dredd would suffice

Options:
  --dry-run, -y                      Do not run any real HTTP transaction, only
                                     parse API description document and compile
                                     transactions.               [default: null]

[alekhrycaiko]

:thinking: I think sort of?

It definitely covers the validation requirements I mentioned. The strange part of that command is that I'd need to then decipher JSON output. I don't believe it returns a 0/1 result depending on success/failure - but perhaps I'm mistaken. That could also be something that could just be built depending on the users needs :man_shrugging: