Linting troubleshoot specs

[banjoh]

Describe the rationale for the suggested feature.

When people craft troubleshoot specs, errors that violate the schemas can go unnoticed. We need a way to lint specs and provide meaningful feedback to users

Describe the feature

• Create a new API to lint specs against troubleshoot schemas that returns results that can be used by clients to render results in UIs etc
• Create a cli subcommand the uses the API above to run against input specs e.g troubleshoot lint <specs>
• ~~Ability to provide linting rules e.g enforce always having timeouts on certain collectors - Do we want this?~~. Lets leave enforcing rules to the OpenAPI schemas

Additional context

• This issue depends on https://github.com/replicatedhq/troubleshoot/issues/871 being implemented first

[chris-sanders]

Linting for validity of a spec seems useful. However, linting shouldn't be used to enforce optional parameters. Linting should only verify that it's valid input. I'm just not sure on the current priority of this. In particular would the linter only process yaml inputs or would you be able to lint a spec in a helm file?

[banjoh]

However, linting shouldn't be used to enforce optional parameters. Linting should only verify that it's valid input.

That's correct.

Linting in this context has emphasis on how validation errors are reported so they can be consumed by clients (CLI, UIs etc)

In particular would the linter only process yaml inputs or would you be able to lint a spec in a helm file?

• For the CLI implementation, I would have the command comform with the other CLIs. Using the helm file example, this would look like helm template <chart> | troubleshoot lint. This would extract troubleshoot specs, lint them against the version of troubleshoot schema of the binary and report errors in a fashion users will know what needs fixing

• For the library implementation, we'd have an API that takes TroubleshootKinds as an input, which would have been loaded prior. The API would then validate the specs against the troubleshoot's OpenAPI schema and return any validation errors. Consumers of this library (including the CLIs) can use this API to report errors.

  func (kinds *TroubleshootKinds) Validate() ([]ValidationError) {
  	...
  }
  

Troubleshoot schemas might need to be packed with the binary. Each version of the binary is strongly dependant on a specific schema.