Repeated revalidate() raises Invalid state

[kputyra]

I've found some weird bug: calling revalidate() for the second time raises an Invalid state exception when optional fields with default values are present in the schema, but not in the YAML document. Here's a snippet to reproduce the problem.

from strictyaml import load, Map, Int, Optional

doc = "x: 1"

# This works fine
schema = Map({ "x": Int(), Optional("y"): Int() })
yaml = load(doc, schema)
try:
    for i in range(1,6):
        yaml.revalidate(schema)
        print(i, end=" ")
except Exception as e:
    print(e)
print("")

# This raises an exception on the second iteration
schema = Map({ "x": Int(), Optional("y", default=42): Int() })
yaml = load(doc, schema)
try:
    for i in range(1,6):
        yaml.revalidate(schema)
        print(i, end=" ")
except Exception as e:
    print(e)

The generated output:

1 2 3 4 5 
1 Invalid state

It seems that the type of the field does not matter: I've got the same behavior for Str and Any.

On the other hand, the exception is not raised when the YAML object is modified, either by removing the optional value or modifying another one i.e. no exception is raised by the following loops

schema = Map({ "x": Int(), Optional("y", default=42): Int(), Optional("z", default=42): Int() })
yaml = load(doc, schema)
try:
    for i in range(1,6):
        yaml['y'] = 18
        del yaml['y']
        yaml.revalidate(schema)

    for i in range(1,6):
        yaml['x'] = i
        yaml.revalidate(schema)

except Exception as e:
    print(e)

It looks to me that revalidate() uses some internal values that are reset whenever the YAML object is modified, but not otherwise.

[aschankler]

It looks like adding default keys scrambles the order of the state of the YAMLChunk object, leading to inconsistency between the ruamel parsing and the strictyaml parsing. This is the same outcome as in #184, where a failed revalidation leads to inconsistency. In the original example:

from strictyaml import load, Map, Int, Optional

doc = "x: 1"
schema = Map({ "x": Int(), Optional("y", default=42): Int() })
yml = load(doc, schema)

yml._chunk.strictparsed()
# ordereddict([(YAML(x), YAML(1)), (YAML(y), YAML(42))])
yml._chunk.contents
# ordereddict([('x', '1')])

yml.revalidate(schema)
yml._chunk.strictparsed()
# ordereddict([(YAML(y), YAML(42)), (YAML(x), YAML(1))])
yml._chunk.contents
# ordereddict([('x', '1')])

The mismatch between these two internal states causes trouble when resolving keys in YAMLChunk.expect_mapping called by Map.validate.

[callegar]

I am also experiencing the issue.

Guess it can be worked around by non revalidating... but...

Otherwise, I wonder if dealing with optional stuff by using a MapCombined and to then explicitly look for some keys and add them with the default values if missing would cause the same issue.

But in fact, would be glad to know if a fix is possible or far away in a pipeline of more urgent stuff...