Nim icon indicating copy to clipboard operation
Nim copied to clipboard

parsecfg does not support raw strings with commas

Open daylinmorgan opened this issue 1 year ago • 2 comments

Description

When parsing a config file with raw text instead of quoted strings a comma starts parsing a new key. This only effects the parser when handling unquoted key/value pairs.

import std/parsecfg
import std/streams

var f = newStringStream("""
key = value
key = value, with comma
""")
var p: CfgParser
open(p, f, "example.ini")
while true:
  var e = next(p)
  case e.kind
  of cfgEof: break
  of cfgSectionStart:   ## a `[section]` has been parsed
    echo "new section: " & e.section
  of cfgKeyValuePair:
    echo "key-value-pair: " & e.key & ": " & e.value
  of cfgOption:
    echo "command: " & e.key & ": " & e.value
  of cfgError:
    echo e.msg
close(p)

If this is intentional behavior than it should probably be noted in the documentation otherwise see below for a possible fix.

Nim Version

Nim Compiler Version 2.0.2 [Linux: amd64] Compiled at 2023-12-15 Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release

Current Output

key-value-pair: key: value
key-value-pair: key: value
key-value-pair: , with comma:

Expected Output

key-value-pair: key: value
key-value-pair: key: value, with comma

Possible Solution

https://github.com/nim-lang/Nim/blob/4b67cccf5097cc5d2a592bf78ae2746cc3ee8959/lib/pure/parsecfg.nim#L220-L221

Adding the , char to this set should probably fix it.

Additional Information

I can submit a PR to patch SymChars or if the behavior is intentional, I can update the documentation to warn users about this edge case,

daylinmorgan avatar Feb 07 '24 01:02 daylinmorgan

The library does not handle bogus cfg format. So if you do this:

key1/test1 = value1
key2_test2 = value2, with comma
key3,test3 = value3

it returns that:

key-value-pair: key1/test1: value1
key-value-pair: key2_test2: value2
key-value-pair: , with comma: 
key-value-pair: key3: 
key-value-pair: ,test3: value3

i would expect an error for invalid key format.

So if you want to use a comma in a value string, enclose it in quote chars. (") adding the "," to SymChars would allow "," in keys.

RokkuCode avatar Feb 07 '24 03:02 RokkuCode

As a configuration format there isn't a formal spec so whether it's bogus or not all depends on what the implementation allows.

It's not clear to me reading the description of the module that commas are disallowed in keys. Also, in previous issues @ringabout has referenced the python implementation which accepts commas both in keys and unquoted values.

So if you want to use a comma in a value string, enclose it in quote chars. (") adding the "," to SymChars would allow "," in keys.

Why can't commas be in keys?

daylinmorgan avatar Feb 07 '24 15:02 daylinmorgan