Feature(schema): Enforce log structure

[EandrewJones]

Problem

Inconsistencies in the schema across versions and between logTypes (interval vs raw) can cause challenges for downstream development/interaction with the logs in both TAP and Distill.

Proposal

Maintain a versioned, default schema that we enforce for all standard log types. Pure custom logs would be the exception. This will guarantee the existence of a core set of key-value pairs in the logs for downstream consumers.

Options:

• JSON schema enforced purely on the UserALE side
• Protobuf/Avro schema that stands apart from the UserALE lib and/or can be shared across all Flagon projects (think Confluent's schema registry)

Risks

• JSON schema is easier but doesn't provide the universal guarantees and type-checking of something like Protobuf/Avro.
• Protobuf/Avro is likely more complicated to implement in a maintainable way across all Flagon repos. Need to research best options.

[Suvendu-UI]

is the issue closed or open ?

[EandrewJones]

This issue is still open.