pyhocon icon indicating copy to clipboard operation
pyhocon copied to clipboard

'include required(file(…))' is broken

Open ElkMonster opened this issue 5 years ago • 2 comments

While include required("file.conf") works as expected, include required(file("file.conf")) raises due to this line (265) in config_parser.py:

path = file if basedir is None else os.path.join(basedir, file)

If the filename to include is wrapped in file(…), the file variable is of type ConfigQuotedString instead of str, so os.path.join() fails.

>>> pyhocon.ConfigFactory.parse_string('include required(file("file.conf"))')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/site-packages/pyhocon/config_parser.py", line 130, in parse_string
    return ConfigParser().parse(content, basedir, resolve, unresolved_value)
  File "/usr/lib/python3.7/site-packages/pyhocon/config_parser.py", line 346, in parse
    config = config_expr.parseString(content, parseAll=True)[0]
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1644, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3446, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3581, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3768, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3970, in parseImpl
    return super(ZeroOrMore, self).parseImpl(instring, loc, doActions)
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3899, in parseImpl
    loc, tokens = self_expr_parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1402, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 3581, in parseImpl
    ret = e._parse( instring, loc, doActions )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1428, in _parseNoCache
    tokens = fn( instring, tokensStart, retTokens )
  File "/usr/lib/python3.7/site-packages/pyparsing.py", line 1072, in wrapper
    ret = func(*args[limit[0]:])
  File "/usr/lib/python3.7/site-packages/pyhocon/config_parser.py", line 271, in include_config
    unresolved_value=NO_SUBSTITUTION
  File "/usr/lib/python3.7/site-packages/pyhocon/config_parser.py", line 77, in parse_file
    with codecs.open(filename, 'r', encoding=encoding) as fd:
  File "/usr/lib/python3.7/codecs.py", line 898, in open
    file = builtins.open(filename, mode, buffering)
TypeError: expected str, bytes or os.PathLike object, not ConfigQuotedString

ElkMonster avatar Oct 25 '18 14:10 ElkMonster

Here's a mwe for the above:

test.conf: { "a" : "Hello", "b" : "World" }

test_include.conf: { include required(file("test.conf")) "c" : ${a}" "${b}"!" }

This fails because of line 317 in config_parser.py: value = final_tokens[1].value if isinstance(token[1], ConfigQuotedString) else final_tokens[1]

I believe it should instead read value = final_tokens[1].value if isinstance(final_tokens[1], ConfigQuotedString) else final_tokens[1]

andreas-thomik avatar Jun 21 '19 08:06 andreas-thomik

Should be fixed by #229.

jrouly avatar May 26 '20 13:05 jrouly