[Fleet] Add hardcoded validation for custom YML in Fleet Server policies

[kpollich]

Ref https://github.com/elastic/fleet-server/issues/1848

When adding custom YML configuration for Fleet Server via its policy settings, we should perform validation to ensure the user can't accidentally send their Fleet Server into an unhealthy state. Allowing custom configuration in this way is great for flexibility, but we should have guard rails in place to prevent a user's entire Fleet environment from going down.

• [ ] Create a validation schema for acceptable values/structure in the Fleet Server policy YML block
• [ ] Add a validation step on the client with inline error reporting
• [ ] Ensure validation also fires on the server to catch validation errors during API requests or preconfiguration

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[nchaulet]

What about instead of hardcoding this? we had a validate json schema to the package spec for yaml field? cc @mtojek @jsoriano

[mtojek]

I don't think that we have validation for the yaml field either: spec. I know that some developers like to place there just a commented blob to show different properties.

[jsoriano]

I don't think that validation of Fleet Server configuration (or any other runtime configuration) belongs to the Package Spec.

Probably the most solid approach for this would be to make Fleet Server rollback to previous healthy configuration if it cannot reach healthy state with the new one. If that happens, it could report something back to Fleet so it can be displayed to the user.

Bringing Fleet Server config validation to Kibana or the Spec is always going to be a catch up game.

[juliaElastic]

Do we have a valid fleet-server config schema stored somewhere or how can we determine if the provided config is valid? Or are we planning to do only a yaml format validation?

[nchaulet]

I don't think that validation of Fleet Server configuration (or any other runtime configuration) belongs to the Package Spec.

@jsoriano I do not think neither but validating the schema of a yaml variable could belong to the package spec no? and I think it's what we need here.

[jsoriano]

I don't think that validation of Fleet Server configuration (or any other runtime configuration) belongs to the Package Spec.

@jsoriano I do not think neither but validating the schema of a yaml variable could belong to the package spec no? and I think it's what we need here.

The package spec is not being used to validate any configuration variable so far, it only defines the types of variables expected. In general, the package spec only defines static information about packages, nothing about runtime validation. Validating runtime configuration and handling runtime problems should belong to runtime components.

There could be something like a json/yaml schema for fleet server configuration, but I guess this would need to be maintained along with fleet server itself, and I don't see how it could help to have it in the package spec.

Maybe an approach could be to allow packages to define schemas for their yaml fields. We would need to:

• Add to the package-spec:
  • Something like a yaml_schema property for vars, so if it has type: yaml, this schema can be used for validation.
  • A new directory for these yamlschema files.
• Add to the fleet_server package a schema file with the schema/s for its yaml fields.
• Make fleet able to install these schemas, and use them to validate the configuration.

But I still think that fleet server (or any agent), should be able to rollback to the last working configuration if it cannot reach a healthy state with the new one. It may be difficult to catch everything through schema validation. For example the config may be perfectly valid, but some endpoint or credential may be wrong, they should be able to revert to working configurations.

[nchaulet]

Add to the package-spec: Something like a yaml_schema property for vars, so if it has type: yaml, this schema can be used for validation.

Yes it was I have in mind, for me it's still validating the type here, and no runtime validation

[kpollich]

Can we file a separate issue in https://github.com/elastic/package-spec for the YML validation and move forward with a "hardcoded" schema specific to the Fleet Server integration in Kibana?