clp icon indicating copy to clipboard operation
clp copied to clipboard

ffi: Add support for auto/user-generated key-value pairs and fix duplicated key detection issues on non-leaf nodes in `KeyValuePairLogEvent`.

Open LinZhihao-723 opened this issue 4 months ago • 1 comments

Description

As planned in #556, we want to differentiate key-value pairs into two categories: auto-generated key-value pairs and user-generated key-value pairs. This PR adds support for auto/user-generated key-value pairs in KeyValuePairLogEvent, where each set of key-value pairs is stored as a set of node-ID-value pairs with a reference to the associated schema tree. However, this PR does not yet support auto/user-generated kv pairs serialization/deserialization. The deserializer will output new KeyValuePairLogEvent instances, but with all kv-pairs as user-generated ones, leaving auto-generated pairs empty.

This PR also fixes a bug where duplicated keys are not correctly detected. Consider the following schema tree:

 <0:root:Obj>
      |
      |------------> <1:a:Obj>
      |                  |
      |--> <2:a:Str>     |--> <3:b:Str>

Node annotation: <nodeID:keyName:valueType>

Before this PR, if we have node-ID-value pairs [<2:"Value0">, <3:"Value1">], it will be considered valid with the schema tree above. However, it is actually invalid since there is an implicit duplicated key under the root node: both node 1 and node 2 have the key `"a". The previous implementation doesn't check key duplications among non-leaf nodes' siblings. This PR fixes this bug by checking key duplications among siblings for all the nodes from the leaf up to the root.

Validation performed

  • Ensure all workflow passed.
  • Ensure existing unit tests all passed with the new KeyValuePairLogEvent that supports auto/user-generated kv pairs.
  • Enhance the current unit tests to verify:
    • Valid inputs can successfully construct an instance of KeyValuePairLogEvent
    • Invalid auto-generated or user-generated node-ID-value pair inputs will be properly captured
    • Both auto-generated and user-generated kv pairs can be correctly serialized
    • The bug mentioned in the Description section is correctly fixed

Summary by CodeRabbit

  • New Features

    • Enhanced serialization of key-value pairs into JSON format.
    • Added validation for uniqueness of node keys among siblings.
    • Improved handling of auto-generated and user-generated schema trees.
  • Bug Fixes

    • Resolved issues with key duplication checks in ancestor nodes.
  • Tests

    • Updated test cases to validate new functionalities and error handling for log event creation.

LinZhihao-723 avatar Oct 14 '24 03:10 LinZhihao-723