ion-hive-serde icon indicating copy to clipboard operation
ion-hive-serde copied to clipboard

Published 20 hours ago • verified publisher icon amzn

Replacing ion struct with Map<> to store key/value fetched from extractor.

Open cheqianh opened this issue 3 years ago • 0 comments

This issue was raised in https://github.com/amzn/ion-hive-serde/pull/59#discussion_r733256794

In ion-hive-serde, we are using an ion struct to store values fetched from a path extractor because of:

  1. We need a key-value store to carry column values
  2. The top-level IonStruct as a context object carries the IonSystem which we use as a ValueFactory in the callbacks created in PathExtractionConfig

  As mentioned in above comment,

Motivation for using a non-IonValue as our top level key-value store and context object is that it intentionally disrupts handling of recursive IonValues. This shouldn't be an IonStruct- it's a collection of columns whose values are IonValues.

It would be good to replace our path extractor context object with a Map<String,IonValue> or some similar non-IonValue context object.

cheqianh avatar Oct 22 '21 22:10 cheqianh