miller icon indicating copy to clipboard operation
miller copied to clipboard

Unflattening doesn't identify arrays when indices are 0-up

Open Veraxus opened this issue 2 years ago • 4 comments

When unflattening a csv to JSON, mlr does not correctly identify arrays by the numeric headers, instead outputting objects with numeric indices.

Input:

things.0,things.1,things.2
apple,orange,watermelon
rivian,subaru,aptera

Command: mlr --c2j cat input.csv > output.json

Output:

[
{
  "things": {
    "0": "apple",
    "1": "orange",
    "2": "watermelon"
  }
},
{
  "things": {
    "0": "rivian",
    "1": "subaru",
    "2": "aptera"
  }
}
]


Expected Output:

[
{
  "things":  [
      "apple",
      "orange",
      "watermelon"
    ]
},
{
  "things":  [
      "rivian",
      "subaru",
      "aptera"
    ]
}
]

Using mlr 6.7.0 on Windows 11 in Powershell 7.3.3

Veraxus avatar Apr 02 '23 20:04 Veraxus

When unflattening a csv to JSON, mlr does not correctly identify arrays by the numeric headers, instead outputting objects with numeric indices.

I think you should start from 1:

things.1,things.2,things.3
apple,orange,watermelon
rivian,subaru,aptera

https://miller.readthedocs.io/en/6.7.0/reference-main-strings/#1-up-indexing

aborruso avatar Apr 02 '23 21:04 aborruso

@Veraxus have you tried it? Does it work for you? If yes, please close the issue

aborruso avatar Apr 08 '23 08:04 aborruso

@aborruso That’s unfortunately not an option/solution since arrays almost always start at 0 and I’m working with automated exports of huge files.

mlr should be able to recognize arrays precisely because they start with a 0 index.

Veraxus avatar Apr 08 '23 16:04 Veraxus

@Veraxus I feel your pain.

I had given deep thought to starting 0-up but after much consideration and reflection I realized that Miller had to be a 1-up system: see also https://miller.readthedocs.io/en/6.7.0/reference-main-arrays/#1-up-indexing

This is fundamental to Miller design.

johnkerl avatar Apr 13 '23 03:04 johnkerl