CC (conversion rule) block is sometimes ignored

[markok-isystem]

Python version

('python=3.8.7 (tags/v3.8.7:6503f05, Dec 21 2020, 17:59:51) [MSC v.1928 64 bit '
 '(AMD64)]')
'os=Windows-10-10.0.19041-SP0'
'numpy=1.23.0'
'asammdf=7.1.0'

Code

MDF version

from asammdf import MDF print(MDF('ARTITasks.mdf').version) 4.11 print(MDF('ARTIUserDatapoints.mdf').version) 4.11

Code snippet

import sys
import asammdf
mdf = asammdf.MDF(sys.argv[1])
for channel in mdf.iter_channels():
  print(channel.name)
  if channel.source:
    print(channel.source.path)
  for timestamp, sample in zip(channel.timestamps, channel.samples):
    print(timestamp, sample)
  print()

Description

The library ignores a channel (CN) block's CC (conversion rule) block if the latter specifies a lookup table for text. This happens with MDF files that are formatted according to the ASAM ARTI 1.0.0 standard. Vector MDF Validator 2.9.9 reports no errors for these files.

In asammdf 6.2.0 and older, the output is:

artiTraceData
AR_CP_OS_TASK
1.0000000000000001e-07 (b'', b'CORE_Core0', b'OsTask_Start', b'MyTaskA')
2.0000000000000002e-07 (b'', b'CORE_Core0', b'OsTask_Preempt', b'MyTaskA')
2.0000000000000002e-07 (b'', b'CORE_Core0', b'OsTask_Start', b'MyTaskB')
3.0000000000000004e-07 (b'', b'CORE_Core0', b'OsTask_Preempt', b'MyTaskB')
3.0000000000000004e-07 (b'', b'CORE_Core0', b'OsTask_Start', b'MyTaskA')

In asammdf 6.3.0 and later, the output is:

artiTraceData
AR_CP_OS_TASK
1.0000000000000001e-07 (b'', 0, 3, 0)
2.0000000000000002e-07 (b'', 0, 1, 0)
2.0000000000000002e-07 (b'', 0, 3, 1)
3.0000000000000004e-07 (b'', 0, 1, 1)
3.0000000000000004e-07 (b'', 0, 3, 0)

For some files (example: ARTIUserDatapoints.mdf), all versions ignore the conversion rules:

artiTraceData
USER_DATAPOINT
1.0000000000000001e-07 (0, 0, 0, 257)
2.0000000000000002e-07 (1, 0, 1, 4294967295)
3.0000000000000004e-07 (2, 0, 2, 1084227584)

The expected output is:

artiTraceData
USER_DATAPOINT
1.0000000000000001e-07 (b'UnsignedData32', b'default', 'uint32', 257)
2.0000000000000002e-07 (b'SignedData32', b'default', 'sint32', 4294967295)
3.0000000000000004e-07 (b'FloatData32', b'default', 'float32', 1084227584)

mdf.zip

[david32]

Hi, I took a look:

In the first file (With Channel AR_CP_OS_TASK), the issue comes from channel type being CHANNEL_TYPE_VALUE and going to the else statement of: https://github.com/danielhrisca/asammdf/blob/1ce34825218b55269815ea5579a4d651135623df/asammdf/blocks/mdf_v4.py#L7222-L7225

In this else statement, there is no conversion call, which is easily fixable.

In the second file (with channel USER_DATAPOINT) The issue seems to come from: _get_structure: https://github.com/danielhrisca/asammdf/blob/1ce34825218b55269815ea5579a4d651135623df/asammdf/blocks/mdf_v4.py#L6714-L6715

The result of that if is true, and following that path, there are no other calls to "get" and the individual channels are not converted (instead, a single conversion of the first channel is applied - which means None.)

@danielhrisca I don't know what this if statement is for in this case, and thus hesitant to make changes. Hopefully some of the information above is helpful and you can fix it or give insights so that I can :)

[danielhrisca]

The result of that if is true, and following that path, there are no other calls to "get" and the individual channels are not converted (instead, a single conversion of the first channel is applied - which means None.)

yes that optimization will not apply the channel conversions for the structure fields, all will have raw values