WIP: Generating a configuration reference

[tarquin-the-brave]

Potential way of generating a structured reference for Floki's config file. Aiming for something akin to API reference docs, e.g. Kubernetes.

This attempted this with Rust Docs.

[rlupton20]

I think with respect to user facing documentation we should strive to document the YAML format that people actually use. Certainly we're nowhere near that at the moment. Some basic organisation of the generated docs would be a start, and a simple reference page.

I'm not against generating rust docs though. I think they're useful for documenting the Rust, and for those who really want to be sure the YAML reference is up to date, but its not a surrogate.

I never really thought much of kubernetes documentation, I just got used to it.

@maxdymond wdyt?

[maxdymond]

Yeah, I think autogenerated docs are nice but the best documentation walks you through common cases, before giving you all the possible options (a lot of the Python documentation on read the docs gets this right).

I think this means we want some curated content, as well as autogenned stuff.

[tarquin-the-brave]

Yeah I totally agree. This is by no means "all the config docs". This is to serve as a complete schema reference.

I'm proposing that we have a "Getting Started Guide", "A Cookbook", etc, that walk through common cases with explanations. This generated reference is then for the 10% of the time where you need to know specifics about the config schema (which currently only source code can tell you).

[tarquin-the-brave]

I've documented each possible field in the config. cargo doc --no-deps & open up in your browser to take a look.

Next step would be to see how the main docs could be refactored into the form of:

• Getting Started
• Cookbook of use cases
  • use case 1: do X + example
  • use case 2: do Y + example
  • etc

Which tbf they're reasonably close to already.

@rlupton20 @max What do we think about the hugo theme used for the docs atm? It's a blog format. It doesn't seem to link you back to the homepage when you click down into "Getting Started". Would something like mdbook be more fitting?

After that I'll setup CI to serve this config reference alongside whatever the main docs become. I serve it of main on my fork so we can see it fitting together in github pages.

[tarquin-the-brave]

I did some experimenting with another way of doing this. Essentially generating a JSON Schema from the config structures then using an existing tool to do JSON Schema -> HTML conversion.

I think given the audience for this aren't guaranteed to be used to Rust it's better to not have the surrounding Rust docs that could cause confusion.