[Bug] Python grass script vector_db_select fails if value contains newline

[marisn]

vector_db_select function reads values returned from v.db.select line by line. This works fine as long as data in any of columns does not contain a newline symbol. If a newline symbol is present, v.db.select output for a single feature spans more than one line. In such case python code tries to interpret each line as a complete one resulting in an IndexError exception on line 289. A solution will be to check if returned line contains expected number of separators. If it contains less, contents of next line should be appended and process repeated. https://github.com/OSGeo/grass/blob/6b61d1087ba9cf7429df8830c636a827bc5c3f5d/python/grass/script/vector.py#L279

[tmszi]

I can't reproduce it:

GRASS nc_basic_spm_grass7/PERMANENT:~ > db.select table=test
cat|name|idx
202|forest|1
1279|urban\n|2
1536|agric\nulture|3

GRASS nc_basic_spm_grass7/PERMANENT:~ > cat /tmp/test.py 
#!/usr/bin/env python3

import grass.script as gs


def main():
    print(gs.vector.vector_db_select('test')['values'])


if __name__ == '__main__':
    main()

GRASS nc_basic_spm_grass7/PERMANENT:~ > python /tmp/test.py 
{202: ['202', 'forest', '1'], 1279: ['1279', 'urban\\n', '2'], 1536: ['1536', 'agric\\nulture', '3']}

[marisn]

It depends on database in use. SQLite does not support backslash escape characters thus output looks like this (yes, that's CHAR(10) instead of \n):

v.db.select map=p1
cat|FID|my_text
1|0|one
two
three

And here is output from your test code:

Traceback (most recent call last):
  File "/home/marisn/rm_test.py", line 11, in <module>
    main()
  File "/home/marisn/rm_test.py", line 7, in main
    print(gs.vector.vector_db_select('p1')['values'])
  File "/home/marisn/soft/grass-marisn/dist.x86_64-pc-linux-gnu/etc/python/grass/script/vector.py", line 289, in vector_db_select
    key_value = int(value[key_index])
ValueError: invalid literal for int() with base 10: 'two'

[wenzeslaus]

The pure vector_db_select is trying to parse the pre-8 format of v.db.select (now still used as the default under the name plain) which is very hard (and impossible to do reliably). The new formats, JSON and true CSV, are much easier to parse and using them is probably the best fix.

Alternatively, with the new formats, one can just skip vector_db_select and use v.db.select directly:

json.loads(gs.read_command("v.db.select", map="viewpoints", columns="cat,height", layer=2, format="json"))["records"]