##TX block unexpectedly converted to ##MD block upon saving file

[markwaterbury]

Python version

('python=3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]')
'os=Windows-10-10.0.19045-SP0'
'numpy=1.26.4'
'asammdf=7.4.1'

Code

MDF version

'4.10'

Code snippet

m=MDF('<name>.mf4')
m.save('<name>_resave.mf4')
m.close()

Traceback

N/A

MDF Validator Error

Invalid XML: Element '{http://www.asam.net/mdf/v4}e' is unexpected according to content model of parent element '{http://www.asam.net/mdf/v4}CNcomment'. Expecting: {http://www.asam.net/mdf/v4}names, {http://www.asam.net/mdf/v4}linker_name, {http://www.asam.net/mdf/v4}linker_address, {http://www.asam.net/mdf/v4}axis_monotony, {http://www.asam.net/mdf/v4}raster, {http://www.asam.net/mdf/v4}formula, {http://www.asam.net/mdf/v4}address, {http://www.asam.net/mdf/v4}common_properties, {http://www.asam.net/mdf/v4}extensions.

Description

When saving a file, whether modifications were made to it or not, asammdf makes some changes which lead to issues in the MDF validator. This only happens when a channel has a valid cn_md_comment text block (id=="##TX"), and that block has a nonempty tx_data field.

What asammdf does:

• If the tx_data field is empty, the cn_md_comment block will not exist in the saved file (address==0x0000). This is not an issue, as the block did not contain any information.
• If the tx_data field is not empty, the text block is changed to a metadata block (from "##TX" to "##MD"). The contents of tx_data are preserved and written to md_data in the new meta data block, however, if these contents are XML (ie. <CNcomment>) they can fail the validation check. Because the comment was previously in a simple text field, it was not subject to this validation. The conversion of the text block to a meta data block is the issue at hand.

[danielhrisca]

Please give a concrete example of the original block and the modified block

[markwaterbury]

Here is an event example. The same behavior can be seen with channel blocks.

Original Event Block (##EV) and ev_md_comment (##TX):

tx_data contents: <EVcomment><TX></TX> <e name="DateTime">2023-11-01 08:32:32 (local = UTC +01:00)</e> <e name="DateTimeUTC">2023-11-01 08:32:32 (UTC)</e> <e name="EventType">PreTrigger</e></EVcomment>

Resaved Event Block (##EV) and ev_md_comment (##MD):

md_data contents: <EVcomment><TX></TX> <e name="DateTime">2023-11-01 08:32:32 (local = UTC +01:00)</e> <e name="DateTimeUTC">2023-11-01 08:32:32 (UTC)</e> <e name="EventType">PreTrigger</e></EVcomment>

Not much difference in the actual event block, other than addresses (resaved file was not compressed, as the original was), but the ev_md_comment blocks show the difference. The data inside the tx/md data blocks remains unchanged, it is simply the reclassification as an XML text metadata block instead of a Text comment block that causes the XML validation to be run (and fail).

[danielhrisca]

If the comment block contains XML then it must be stored as ##MD

[markwaterbury]

When I read the MDF spec (5.7-5.8), it seems the inverse is true: if the block is ##MD, it must contain XML. But the ##TX block can have any UTF-8 text in it.

Additionally, it states:

if explicitly a TXBLOCK is required, it cannot be replaced by an MDBLOCK

As the tool wrote these as text blocks, I do not think they should have been replaced with meta data blocks, even if the text in them is XML.

[danielhrisca]

Which tool wrote the file originally? It looks like a dumb way to write the XML

[markwaterbury]

ATI VISION. I don't disagree with you, but it does appear to be valid.

[danielhrisca]

You should report an issue to ATI VISION. Custom e tags must be places inside the common_properties tag

[markwaterbury]

Again, I don't disagree with you, but if the text is not in an MDBLOCK it should not be interpreted as XML. So it seems like the TXBLOCK should be able to have anything in it, XML or not, regardless of XML schema.

[Georgeeoo]

please，could you tell me how to change METADATA?

[Georgeeoo]

vector validator only can watch METADATA, but now I dont know how to change it.