Scope merging

[kenodegard]

The following is valid however tomlkit isn't able to properly handle the dual scopes:

a.b.c = 12

[a.b]
d = 34

import tomlkit

doc = tomlkit.parse("""\
a.b.c = 12

[a.b]
d = 34
""")

# looking at the doc shows incomplete data
doc  # {'a': {'b': {'d': 34}}}

# retrieving data works
doc['a']['b']['c']  # 12

# modifying data fails
doc['a']['b']['c']  = 45
doc['a']['b']['c']  # 12

doc['a']['b']['d']  = 100
doc['a']['b']['d']  # 34

[eksortso]

Two important observations need to be made.

First, I ran the same code, and observed some other properties. This points to what could be considered a bug, but not the one you are thinking of. (I trimmed some extraneous blank lines.)

>>> doc
{'a': {'b': {'d': 34}}}

>>> doc['a']
{'b': {'d': 34}}

>>> doc['a']['b']  # Well, look at that! The c is back.
{'c': 12, 'd': 34}

>>> tomlkit.__version__
'0.5.3'

>>> sys.version_info
sys.version_info(major=3, minor=7, micro=1, releaselevel='final', serial=0)

Second, the document is not valid TOML, because the contents of a.b are split between two sections: the root section at the top, and the [a.b] section. You cannot define any table more than once, and the table a.b is defined when a.b.c = 12 is specified. As further proof, I used the toml library to parse the document, and got this error:

What? b already exists?{'b': {'c': 12}} (line 3 column 1 char 12)

The fact that tomlkit does not raise an error when parsing the document is the bug.

[kenodegard]

Ok, I don't have an issue with this viewpoint I just can't find the TOML specs explicitly stating that implicit and explicit tables cannot be mixed. Just checked the ABNF and it doesn't prohibit it (but then the ABNF permits the same table to be defined multiple times and I know that's not permitted).

Just checked qTOML, they allow this.

[eksortso]

It was a matter of interpretation that wasn't explicitly specified in the standard, though the principle that you cannot define any table more than once is explicitly stated in v0.5.0.

The introduction of dotted keys opened up a can of worms; there is still interpretation to be applied, and standards language to be updated. Based on the strictest standards possible, the TOML in your example is invalid because the table a.b is introduced (implicitly by assigning a value to a.b.c) in the root table block before its own table block is introduced. I'm inclined to believe that the same restriction would apply under looser interpretations.

I've brought this up for debate under the TOML standard issue where these interpretations are being made.

By the way, the ABNF would consider the example well-formed, but it does nothing to identify whether it's valid.

[kenodegard]

Ok, so I got caught up on the toml discussion, and I agree this isn't explicitly understood yet.

At least until this is better defined I feel the correct answer to this is to leave it up to the developer using tomlkit to decide which interpretation to allow in their case.