pyhocon icon indicating copy to clipboard operation
pyhocon copied to clipboard

Quoted keys aren't output with quotes, problem since pyhocon keys are limited

Open movermeyer opened this issue 8 years ago • 0 comments

Given the following config:

foo: {
    <bar>: bar
}

And the code:

from pyhocon import ConfigFactory
config1 = ConfigFactory.parse_file("test.conf")

You get an Exception.

Traceback (most recent call last):
  File "test.py", line 6, in <module>
    config1 = ConfigFactory.parse_file("test.conf")
  File "env/lib/python2.7/site-packages/pyhocon/config_parser.py", line 48, in parse_file
    return ConfigFactory.parse_string(content, os.path.dirname(filename), resolve)
  File "env/lib/python2.7/site-packages/pyhocon/config_parser.py", line 87, in parse_string
    return ConfigParser().parse(content, basedir, resolve)
  File "env/lib/python2.7/site-packages/pyhocon/config_parser.py", line 269, in parse
    config = config_expr.parseString(content, parseAll=True)[0]
  File "env/lib/python2.7/site-packages/pyparsing.py", line 1632, in parseString
    raise exc
pyparsing.ParseSyntaxException: Expected "}" (at char 11), (line:2, col:5)

This itself might be a bug. I don't see anywhere in either the HOCON spec, or the JSON syntax that says that angle brackets are not allowed in keys.

Digging into the code:

pyhocon doesn't define a key in the same way as JSON/HOCON does. JSON allows nearly any Unicode character, but pyhocon only allows a subset of ASCII characters. Outside of those characters, the key has to be double-quoted. Code here

However, even if it is not a bug, it still leads to odd behaviour. I wrapped the key in double quotes:

foo: {
    "<bar>": bar
}

But when I "round-trip" the config file, parsing it and writing it back out:

from pyhocon import ConfigFactory
from pyhocon.tool import HOCONConverter

config1 = ConfigFactory.parse_file("test.conf")
with open("test2.conf", 'wb') as fout:
    fout.write(HOCONConverter.to_hocon(config1))

The output is the original failing HOCON file since it did not double-quote the keys on output:

foo: {
    <bar>: bar
}

Which of course fails when I try to parse it in the next script that uses pyhocon.

I think this falls into one of two cases:

  • pyhocon should support the full range of acceptable JSON/HOCON keys
  • HOCONConverter should double-quote keys containing angle brackets on output to HOCON

movermeyer avatar Oct 04 '17 13:10 movermeyer