arrow2 icon indicating copy to clipboard operation
arrow2 copied to clipboard

Published 20 hours ago • verified publisher icon jorgecarleitao

`json::infer` inconsistent with `ndjson::infer`

Open universalmind303 opened this issue 2 years ago • 1 comments

Arrow version: latest rust version: rustc 1.64.0-nightly (0f4bcadb4 2022-07-30)

The ndjson::infer method takes in a reader, and num_rows as seen here

json::infer only takes in a json_deserializer::Value

It would be a nice improvement if the api's were consistent, and you could easily infer schemas for json arrays as well as ndjson.

universalmind303 avatar Aug 04 '22 21:08 universalmind303

The main reason is that it is possible to infer from ndjson in a streaming fashion (item by item), but it is usually not possible to do it with json, since we need the whole object.

But maybe there is a better API?

jorgecarleitao avatar Aug 04 '22 21:08 jorgecarleitao