Comments immediately after section start cause KeyError exception

[rbray89]

test_str in test_comment_preserve_decoder_encoder can be modified to the following to trigger the issue

def test_comment_preserve_decoder_encoder():
    test_str = """[[products]]
# First comment
name = "Nail"
sku = 284758393
# This is a comment
color = "gray" # Hello World
# name = { first = 'Tom', last = 'Preston-Werner' }
# arr7 = [
#  1, 2, 3
# ]
# lines  = '''
# The first newline is
# trimmed in raw strings.
#   All other whitespace
#   is preserved.
# '''
[animals]
# another First comment
color = "gray" # col
fruits = "apple" # a = [1,2,3]
a = 3
b-comment = "a is 3"
"""

[jvail]

I think I have the same issue. Apparently dicts are not allowed to have comments. I would like to add comments (and preserve them) on each level. Not sure if this is a bug or intended. I get a KeyError as well:

test_str = """
[a_dict]  # a dict
val = 'val' # a value
"""
toml.loads(test_str, decoder=toml.TomlPreserveCommentDecoder())

[uiri]

Hi!

Thank you for reporting this issue. I am having trouble reproducing it.

Do you know which version of toml you were using to trigger this issue? Is it possible that it has been fixed in the development branch since the latest release?

[jvail]

Hi,

I get the error with the current version:

import toml
print(toml.__version__)
test_str = """
[a_dict]  # a dict
val = 'val' # a value
"""
toml.loads(test_str, decoder=toml.TomlPreserveCommentDecoder())

0.10.1
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-5-ce7352daeb49> in <module>
      5 val = 'val' # a value
      6 """
----> 7 toml.loads(test_str, decoder=toml.TomlPreserveCommentDecoder())

~/miniconda3/envs/xarray-fspm/lib/python3.8/site-packages/toml/decoder.py in loads(s, _dict, decoder)
    372             pos += len(s[idx - 1]) + 1
    373 
--> 374         decoder.embed_comments(idx, currentlevel)
    375 
    376         if not multilinestr or multibackslash or '\n' not in multilinestr:

~/miniconda3/envs/xarray-fspm/lib/python3.8/site-packages/toml/decoder.py in embed_comments(self, idx, currentlevel)
   1049 
   1050         key, comment, beginline = self.saved_comments[idx]
-> 1051         currentlevel[key] = CommentValue(currentlevel[key], comment, beginline,
   1052                                          self._dict)

KeyError: ''

[andrewwade]

At first I overwrote the TomlPreserveCommentDecoder class's embed_comments() function to first check if the key exists in currentlevel. Which fixes this issue, but comments at the section level are lost. Comments are only attached to the last key-value pair that was read. So really it only supports inline comments. The value in the key-value pair gets replaced by an object which contains the real value and a comment. I recommend we do the same for sections. See the following sudo code example.

[section1]
# This is section 1
key1 = 'val' # a value
key2 = true # second value
[section2] # section 2 inline comment
# section 2 comment line
key1 = 1.0 # a section2 value

The behavior currently looks like this (if we check that the key exists):

>>>print config['section1']['key2'].comments
second value
section 2 inline comment
section 2 comment line

Notice the comment in section 2 is stored with the last key of section 1. But it should look something like this:

>>>print config['section1']['key2'].comments
second value
>>>print config['section2].comments
inline comment
section 2 comment line