flagon icon indicating copy to clipboard operation
flagon copied to clipboard

Feature(schema): Enforce log structure

Open Jyyjy opened this issue 9 months ago • 0 comments

Copied from https://github.com/apache/flagon-useralejs/issues/443

Problem

Inconsistencies in the schema across versions and between logTypes (interval vs raw) can cause challenges for downstream development/interaction with the logs in both TAP and Distill.

Proposal

Maintain a versioned, default schema that we enforce for all standard log types. Pure custom logs would be the exception. This will guarantee the existence of a core set of key-value pairs in the logs for downstream consumers.

Options:

JSON schema enforced purely on the UserALE side
Protobuf/Avro schema that stands apart from the UserALE lib and/or can be shared across all Flagon projects (think Confluent's schema registry)

Risks

JSON schema is easier but doesn't provide the universal guarantees and type-checking of something like Protobuf/Avro.
Protobuf/Avro is likely more complicated to implement in a maintainable way across all Flagon repos. Need to research best options.

Jyyjy avatar Apr 04 '25 23:04 Jyyjy