kaitai_struct_compiler
kaitai_struct_compiler copied to clipboard
Python/Construct: Use `unsigned char` representation when converting to `bytes`
Currently Python (and Construct) KST test tests\formats\expr_bytes_non_literal.ksy
meta:
id: expr_bytes_non_literal
seq:
- id: one
type: u1
- id: two
type: u1
instances:
calc_bytes:
value: '[one, two].as<bytes>'
failed with the following error:
Traceback (most recent call last):
File "/tests/spec/construct/test_expr_bytes_non_literal.py", line 9, in test_expr_bytes_non_literal
r = _schema.parse_file('src/enum_negative.bin')
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 309, in parse_file
return self.parse_stream(f, **contextkw)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 300, in parse_stream
return self._parsereport(stream, context, "(parsing)")
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 312, in _parsereport
obj = self._parse(stream, context, path)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 1982, in _parse
subobj = sc._parsereport(stream, context, path)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 312, in _parsereport
obj = self._parse(stream, context, path)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 2440, in _parse
return self.subcon._parsereport(stream, context, path)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 312, in _parsereport
obj = self._parse(stream, context, path)
File "/usr/local/lib/python2.7/site-packages/construct/core.py", line 2576, in _parse
return self.func(context) if callable(self.func) else self.func
File "/tests/compiled/construct/expr_bytes_non_literal.py", line 8, in <lambda>
'calc_bytes' / Computed(lambda this: struct.pack('2b', this.one, this.two)),
error: byte format requires -128 <= number <= 127
The reason is because PythonTranslator uses signed char representation of bytes when casting to bytes type using struct.pack('Nb'), but it tries to pack from u1 (unsigned) integers. So 0xFF from test data raises an error.
It is unclear, what to do here. I speculate, that we expect that arrays for conversion must contain integers in the range [0; 255] and implement it in my PR, but that is only my assumption. I suspect is it correct, although, because both Perl and Ruby uses 'C*' which in both means unsigned chars.
Also, I think, we should add tests with s1 types, alone and mixed with u1 types. What the expected result of converting arrays of such values to bytes type?