pkl icon indicating copy to clipboard operation
pkl copied to clipboard

Duplicate Key Issue When Converting pkl to JSON

Open lis186 opened this issue 1 year ago • 2 comments

Description

When using pkl for object definitions and converting them to JSON format, I encountered an issue where duplicating nested object entries with the same entry names results in generated JSON with duplicate keys, violating JSON standards.

Steps to Reproduce

Here is the pkl code defining a series of beverages and their ingredients:

Tea {
  Base = "Black Tea"
}

BubbleTea = (Tea) {
  Additives {
    "Pearls"
  }
}

MilkBubbleTea = (BubbleTea) {
  Additives {
    "Creamer"
  }
}

FreshMilkBubbleTea = (BubbleTea) {
  Additives {
    "Fresh Milk"
  }
}

FreshMilkBubbleTea2 = (MilkBubbleTea) {
  ["Additives"] {
    "Fresh Milk"
  }
}

Expected Behavior

When converting to JSON, the process should detect potential duplicate keys resulting from nested object entries with the same name and either halt with an error or provide a warning. This behavior would prevent the generation of invalid JSON formats and ensure data integrity, especially in scenarios involving object inheritance and overriding.

Actual Behavior

The generated JSON for the FreshMilkBubbleTea2 object includes duplicate Additives keys, as shown below:

{
  "Tea": {
    "Base": "Black Tea"
  },
  "BubbleTea": {
    "Base": "Black Tea",
    "Additives": [
      "Pearls"
    ]
  },
  "MilkBubbleTea": {
    "Base": "Black Tea",
    "Additives": [
      "Pearls",
      "Creamer"
    ]
  },
  "FreshMilkBubbleTea": {
    "Base": "Black Tea",
    "Additives": [
      "Pearls",
      "Fresh Milk"
    ]
  },
  "FreshMilkBubbleTea2": {
    "Base": "Black Tea",
    "Additives": [
      "Pearls",
      "Creamer"
    ],
    "Additives": [
      "Fresh Milk"
    ]
  }
}

This results in an invalid JSON format since JSON does not allow duplicate keys at the same level.

lis186 avatar Feb 04 '24 08:02 lis186

Interesting bug! I think this is happening because your examples are Dynamic objects, which can contain not just properties (as one might think), but also "entries" and "elements" - meaning a Pkl Dynamic object can be treated like a key-value collection, a list/array-style collection, and a object with properties all at the same time.

In a non-trivial applications, you'd probably want to use Mapping for key-value collections, Listings for array-style collections, typed objects for known structures, and Dynamic almost never.

That's not to say I don't think this is a bug! Of course Pkl's JSONRenderer should not produce invalid output.

jasongwartz avatar Feb 04 '24 13:02 jasongwartz

@jasongwartz is correct: This is because Dynamics include separate entries and properties, whereas JSON does not. I think I agree this is buggy default behaviour for JsonRenderer and that it should at least be a configurable option. I'm not entirely sure, though; is it reasonable to expect this validation when avoiding types? For Typed objects, this already isn't a problem, because it's not expressible.

holzensp avatar Feb 05 '24 13:02 holzensp