Inherit explicit null as override when decoding json/yaml

[steeling]

json.decode doesn't respect nulls properly.

The following example does work as expected:

schema Temp:
    x: int
    y?: int = x

correct: Temp = {x = 1} | {y = None}
incorrect: Temp = {x = 1} | json.decode('{"y": null}')

result:

correct:
  x: 1
  y: null

incorrect:
  x: 1
  y: 1

[Peefy]

The default attribute operator of the config is : that came from the external config. Thus, 1 | None = 1. If we want to merge like the y = None, we can use the json_merge_patch lib like: https://www.kcl-lang.io/docs/user_docs/support/faq-kcl#use-the-json_merge_patch-module-to-merge-configuration

[steeling]

We can actually further simplify the request here, which should include this scenario:

m: Temp = json.decode('{"x": 1, "y": null}')

Coming from raw JSON there is no way to explicitly disable optional (but defaulted) values in a schema.

[steeling]

having some option like json.decode('{}', convert_null_to_none=True) would be nice

[steeling]

This actually yields some odd behavior:

schema MyObj:
    x: int  
    y?: int = 9

temp: MyObj = json.decode('{"x": 1, "y": null}') 

schema Child:
    parent: MyObj = {y = 2}


output = Child{
    parent: temp
}

The above does not yield the same results as:

schema MyObj:
    x: int  
    y?: int = 9

schema Child:
    parent: MyObj = {y = 2}


output = Child{
    parent:  json.decode('{"x": 1, "y": null}') 
}

[Peefy]

This is as expected, as for schema, its attribute operator is assumed to be '=' by default. The former is equivalent to merging schema parent and schema temp, while the latter is equivalent to merging schema parent and dict.

[steeling]

Gotcha that makes sense thanks for the clarification! And as always thanks for the prompt responses :). In regards to this question Is there anyway to cast it to the type I want? Or a way I can do this with a plugin? Or is this something that might be supported in the future?

If willing I’m happy to open a PR related to this if there’s a clear path forward. Maybe passing an optional type to the decode functions?

[Peefy]

In KCL, they are done automatically and do not require additional type for the decode function.

import json

schema SomeSchema:
    foo: str

    check:
        len(foo) >= 3

# Inst is a schema that satisfies the check condition instead of a dynamic config
inst: SomeSchema = json.decode('{"foo": "bar"}')

[steeling]

Hmm, so you're saying now that it returns a type, but before you mentioned it returns a dict? In either case there seems to be no current solution to what I'm looking to do?

Specifically I want to consume yaml where a user can set a value to null, even if the value in the schema has a default. I think the below example shows the problem more clearly:

import json

schema SomeSchema:
    foo?: str = "default"


obj1 = json.decode('{"foo": null}')
obj2 = {foo = None}

inst1: SomeSchema = obj1
inst2: SomeSchema = obj2



val = obj1 == obj2

yields:

obj1:
  foo: null
obj2:
  foo: null
inst1:
  foo: default
inst2:
  foo: null
val: true

This shows 2 objects that are equal but have different behavior. This seems like a bug to me, or at the very least a feature deficit. A simple solution could be adding an optional param to decode to enforce explicit nulls.