[BUG] GetJsonObject fails at spark unit test $.store.book[*].reader

[revans2]

Describe the bug In the spark unit test for GetJsonObject with a path of $.store.book[*].reader and data of

  val json =
    """
      |{"store":{"fruit":[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}],
      |"basket":[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]],"book":[{"author":"Nigel Rees",
      |"title":"Sayings of the Century","category":"reference","price":8.95},
      |{"author":"Herman Melville","title":"Moby Dick","category":"fiction","price":8.99,
      |"isbn":"0-553-21311-3"},{"author":"J. R. R. Tolkien","title":"The Lord of the Rings",
      |"category":"fiction","reader":[{"age":25,"name":"bob"},{"age":26,"name":"jack"}],
      |"price":22.99,"isbn":"0-395-19395-8"}],"bicycle":{"price":19.95,"color":"red"}},
      |"email":"amy@only_for_json_udf_test.net","owner":"amy","zip code":"94025",
      |"fb:testid":"1234"}
      |""".stripMargin

we return [[{"age":25,"name":"bob"},{"age":26,"name":"jack"}]], but the CPU returned [{"age":25,"name":"bob"},{"age":26,"name":"jack"}]

So we have an extra layer of wrapping in a list compared to Spark. I believe that this might be a bug in Spark as https://jsonpath.com/ returns the same result as us, but because this comes directly from a unit test in Spark we need to understand better before we file any Spark issues.

[revans2]

Probably want to see what happens if this matched more than a single list of readers.

[revans2]

After looking into this some more I think this is a bug in Spark and I filed an issue for it at https://issues.apache.org/jira/browse/SPARK-46778

But I don't think that makes our job easier, as we need to be bug for bug compatible with Spark, or we would need to fall back to the CPU for any json path with a [*] in it by default.

[res-life]

Will be fixed by: https://github.com/NVIDIA/spark-rapids-jni/pull/1868