Fiona icon indicating copy to clipboard operation
Fiona copied to clipboard

Field Name encode error when writing to Mapinfo File

Open alexJhao opened this issue 8 months ago • 7 comments

Expected behavior and actual behavior.

I read data from GPKG, then write data to a Mapinfo TAB file. I found the field name error encode.

>>> with fiona.open(gpkg_fp,'r') as src:
...     driver=src.driver
...     crs=src.crs
...     schema=src.schema
...     feat=src[1]
... 
>>> driver
'GPKG'
>>> crs
CRS.from_epsg(4326)
>>> schema
{'properties': {'地市': 'str:80', '区县': 'str:80', '商业街名称': 'str:80'}, 'geometry': 'Polygon'}

>>> with fiona.open(r'e:\temp\7\a.tab','w',driver='MapInfo File',crs=crs,schema=schema) as dst:
...     dst.write(feat)
... 
>>> with fiona.open(r'e:\temp\7\a.tab','r') as dst1:  
...     new_schema=dst1.schema
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "D:\miniconda3\envs\geo_py3b5\Lib\site-packages\fiona\collection.py", line 293, in schema
    self._schema = self.session.get_schema()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "fiona\\ogrext.pyx", line 761, in fiona.ogrext.Session.get_schema
UnicodeDecodeError: 'gbk' codec can't decode byte 0xd0 in position 3: incomplete multibyte sequence

I use binary mode to open the a.tab file. found the field name changed.

>>> with open(r'e:\temp\7\a.tab','rb') as f:
...     context=f.read()
... 
>>> context
b'!table\n!version 300\n!charset Neutral\n\nDefinition Table\n  Type NATIVE Charset "Neutral"\n  Fields 3\n    _\xd8\xca\xd0 Char (80) ;\n    \xc7\xf8\xcf\xd8 Char (80) ;\n    \xc9\xcc\xd2__\xd6\xc3\xfb_\xc6 Char (80) ;\n'

in the TAB file, three field names is:

_\xd8\xca\xd0
\xc7\xf8\xcf\xd8
\xc9\xcc\xd2__\xd6\xc3\xfb_\xc6

but the original field name are: '地市', '区县', '商业街名称' their 'ansi' code are :

>>> [x.encode('ansi') for x in ['地市','区县','商业街名称']]
[b'\xb5\xd8\xca\xd0', b'\xc7\xf8\xcf\xd8', b'\xc9\xcc\xd2\xb5\xbd\xd6\xc3\xfb\xb3\xc6']

b'\xb5\xd8\xca\xd0' become  b'_\xd8\xca\xd0'
b'\xc9\xcc\xd2\xb5\xbd\xd6\xc3\xfb\xb3\xc6'  become  b'\xc9\xcc\xd2__\xd6\xc3\xfb_\xc6'

Operating system

Win10

Fiona and GDAL version and provenance

python 3.11.5 GDAL 3.6.2 fiona 1.9.6

alexJhao avatar May 30 '24 15:05 alexJhao