kaitai_struct_compiler icon indicating copy to clipboard operation
kaitai_struct_compiler copied to clipboard

Construct: use `lambda this:` for ternary expressions

Open Mimickal opened this issue 3 years ago • 0 comments

Construct does not support using conditionals in this expressions. Currently, the compiler outputs ternary expressions like this: ((this.len + 1) if this.len > 0 else 0). That expression does not evaluate the way you might expect. I expect that when this.len is 0, the value supplied to Construct is also 0. The value that actually gets supplied currently is 1.

I've cooked up a practical demo of the problem.

I'm using this ksy:

meta:
  id: ternary
seq:
  - id: len
    type: u1
  - id: data
    type: str
    size: 'len > 0 ? len + 1 : 0'
    encoding: ASCII

This test script contains the Construct struct the compiler outputs before and after this change. It feeds several values through the old and new struct to demonstrate the behavior I'm correcting

# Demonstrates that Construct requires a lambda to access `this` in conditional contexts
from construct import *
from construct.lib import *

# Output from ksc before this change
ternary_old = Struct(
	'len' / Int8ub,
	'data' / FixedSized(((this.len + 1) if this.len > 0 else 0), GreedyString(encoding='ASCII')),
)

# Output from ksc after this change
ternary_new = Struct(
	'len' / Int8ub,
	'data' / FixedSized((lambda this: (this.len + 1) if this.len > 0 else 0), GreedyString(encoding='ASCII')),
)

for val in [
    b'\x00',  # Old fails because it still expects 1 char even though it
              # should expect 0. New handles this properly.
    b'\x01a', # Both fail, since we expect 2 chars but only get 1.
    b'\x01aa' # Both succeed, since we expect 2 chars and get 2 chars.
]:
    try:
        print('old', val, ternary_old.parse(val), '\n')
    except Exception as e:
        print('old', val, str(e), '\n')

    try:
        print('new', val, ternary_new.parse(val), '\n')
    except Exception as e:
        print('new', val, str(e), '\n')

This script produces the following output:

old b'\x00' Error in path (parsing) -> data
stream read less than specified amount, expected 1, found 0 

new b'\x00' Container: 
    len = 0
    data = u'' (total 0) 

old b'\x01a' Error in path (parsing) -> data
stream read less than specified amount, expected 2, found 1 

new b'\x01a' Error in path (parsing) -> data
stream read less than specified amount, expected 2, found 1 

old b'\x01aa' Container: 
    len = 1
    data = u'aa' (total 2) 

new b'\x01aa' Container: 
    len = 1
    data = u'aa' (total 2) 

Mimickal avatar Jun 11 '22 04:06 Mimickal