JsonSource should support path expressions

[hannesmiller]

trafficstars

JsonSource should support path expressions

Scenario

• If I have defined a structure like the following:

struct Person {
     name:String,
     age: Int,
     sex: String
}

• I should be able to specify a projection like so:

withProjection("person.name", "person.age")

• The above returns scalar values for name and age.
• Likewise I think it should work for withPredicate
• Jackson has has processor for drilling into nested types using path expressions which are similar to XPath
• You could put forward a case that this syntax should be the language for other formats that support nested types, Avro, Parquet and Orc

[sksamuel]

It might make sense to do this on the frame level so all sources can benefit. The advantage to source level is when a source supports pushdowns but that's only parquet and orc.

[hannesmiller]

Yeah good point!

Here's a good resource: http://goessner.net/articles/JsonPath/