mdfreader
mdfreader copied to clipboard
ratal
Can not read .mf4 file
Hi Ratal!
Firstly, thanks for the mdfreader module :) So, I have an issue. I cannot read a mf4 file. The file has been validated with MDFValidator (Vector). Tried to run it in Linux environment also.
Code: from mdfreader import mdf4
my_object = mdf4() my_object.read4(r'c:\temp\jonas\Epic.mf4')
Output:
Traceback (most recent call last):
File "C:/code_projects/mdfreader/main.py", line 4, in
Thanks in advance!
Hi Meidland,
I lately made modification to allow opening directly from submodule (in master, you might use released version)
However, I would recommend you to open it using main module mdfreader, not mdf4reader:
from mdfreader import mdf my_object=mdf()
Version of file will be detected and correct submodule used.
Hi again!
Now it worked better :) But I'm still having some issues.
Input:
from mdfreader import mdf
my_object = mdf() my_object.read('my_file.mf4')
Output:
C:\code_projects\mdfreader\mdfenv\Scripts\python.exe C:/code_projects/mdfreader/main.py Unexpected error: (<class 'ImportError'>, ImportError("No module named 'dataRead'",), <traceback object at 0x03DA9530>) dataRead crashed, back to python data reading Unexpected error: (<class 'ImportError'>, ImportError("No module named 'dataRead'",), <traceback object at 0x03DAC5D0>) dataRead crashed, back to python data reading Unexpected error: (<class 'ImportError'>, ImportError("No module named 'dataRead'",), <traceback object at 0x03DB07B0>) dataRead crashed, back to python data reading Unexpected error: (<class 'ImportError'>, ImportError("No module named 'dataRead'",), <traceback object at 0x03D93080>) dataRead crashed, back to python data reading Traceback (most recent call last): File "C:\code_projects\mdfreader\mdfenv\lib\site-packages\mdfreader\mdf4reader.py", line 1200, in readBitarray from dataRead import dataRead ImportError: No module named 'dataRead'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/code_projects/mdfreader/main.py", line 4, in
Hi Meidland,
Your last lines are duplicated and I do not have final error line. But I guess this is this code : self[chan].CFormat.unpack(temp[i].tobytes())[0]
I supposed the temp data type does not match the CFormat. Maybe you can print it to confirm ?
Regarding dataRead cython extension, it is apparently not compiled properly.
I guess you compiled dataRead and solved your issue (the python backup is not as robust) As no feed back since a while closed but do not hesitate to reopen
I'm having a similar issue. Don't know the exact problem. The same code works for another file.
Traceback (most recent call last):
File "C:\Users\mousavin\workspace\Clustering\run.py", line 19, in
I fixed it myself, but I'm not sure if my solution is correct.
For some reason mdfinfo3.py", line 590, in read_tx_block fid.read(4) returns an empty byte.
An if condition fixes the problem.
def read_tx_block(fid, pointer):
""" reads text block """
if pointer != 0 and pointer is not None:
fid.seek(pointer)
read_bytes = fid.read(4)
if read_bytes: # <--- I added this line
(block_type,block_size) = tx_struct.unpack(read_bytes)
text = unpack('{}s'.format(block_size - 4), fid.read(block_size - 4))[0]
return text.rstrip(b'\x00').decode('latin1', 'replace') # .encode('utf-8')
Dunno, if my file is corrupt or not. Could you please clarify.
Edit: fid.read(4) should not be called twice
@Nimi42 Hi, can you send the file (you can find my email on my profile page).
Looking in internet, empty return from read would mean either file is empty or pointer is at end of file. Can you print pointer and check it is within file ? You can confirm by reading it with MDFValidator from Vector. Or you can send the file as proposed Daniel. Be carefull with your modification, calling 2 times fid.read(4) consecutively will create lot of issues as it will not return the same portion of file.
First of all,
thank you for your quick reply.
@ratal Yes. I just realised. It does not fix the problem though. The file can be read with this work around. The labels are correct, but the values are not.
I might add that it is possible to open up the file with MDA.
Since I could not find your mail address I did not send you the file though.
@danielhrisca I send you the file via mail.
Edit: Seems like the mdf version is very old. Something like 214. Is the mdfreader module able to handle these old version?.
@ratal I think mdfreader reads 228 bytes for CNBLOCK. In .dat version 2.14 the CNBLOCK only has 222 bytes (there is no display name and additional byte offset).
3.10 file
Well unfortunately I can't change the format in the recording tool, because I am not the one recording the files.
Also asammdf could not handle the 2.14 version either.
It is possible to use MDA to convert the files, and use asammdf or mdfreader to read them with python, but that is rather tedious.
I was hoping the libs would be able to handle it.
@Nimi42 I was able to parse the file that you have send over email with the code in the development branch. Do you have problems with other files?
@ratal Thank you very much
@danielhrisca I tried the module
from asammdf import MDF
reader = MDF(src)
reader.select(['VehV_v'])
throws an exception with asammdf (2.7.1):
File "C:\Users\mousavin\AppData\Roaming\Python\Python36\site-packages\asammdf\mdf.py", line 59, in __init__
raise MdfException(message.format(name, version))
asammdf.utils.MdfException: "C:\AppData\MF_RDE-Bewertung\Daten\20171020_HN-LI8220_Stadt_00001.dat" is not a supported MDF file; " " file version was found
Edit: I might add that I'm a little bit confused by the API, because in the code it seems like filter and select do the same thing.
@Nimi42 I've added initial support for MDF version 2.14 to the development branch https://github.com/danielhrisca/asammdf/tree/development/asammdf . I think issues not related to mdfreader should continue on the asammdf issue tracker.
@danielhrisca
Oh yes I see. But the values are still not correct. The same as with my quick fix above. It does not seem to be that easy to add support for the old version.
I'm sorry for the trouble.
np.set_printoptions(threshold=np.nan)
reader = MDF(os.path.join(root, f))
signal = reader.select(['Zeit'])
print(signal[0].samples)
Prints:
[ 1.08697600e+03 4.26298573e+04 3.08700774e+04 0.00000000e+00
0.00000000e+00 0.00000000e+00 ... ]
But it should be:
[ 4.246 5.238 6.238 7.238 8.23799 9.23799 ... ]
Similarly with mdfreader
But it should be:
[ 4.246 5.238 6.238 7.238 8.23799 9.23799 ... ]
Similarly with mdfreader
What application did you use to confirm those values?
@Nimi42 You're right. CANape shows correct values. ~~It must be something strange with the data layout in version 2 (compared to versions 3 and 4)~~
Currently mdfreader is not able to read version 2.x of MDF. As it seems relatively simple from 3.x, it could be implemented but it will require a bit of time.
Hello,
I have a similar Problem that i get an error code when i read my mf4 file. Is there any solution to this?
StopIteration Traceback (most recent call last)
/usr/local/lib/python3.6/site-packages/asammdf/mdf.py in init(self, name, memory, version, callback) 96 self._mdf = MDF3(name, memory, callback=callback) 97 elif version in MDF4_VERSIONS: ---> 98 self._mdf = MDF4(name, memory, callback=callback) 99 elif version in MDF2_VERSIONS: 100 self._mdf = MDF2(name, memory, callback=callback)
/usr/local/lib/python3.6/site-packages/asammdf/mdf_v4.py in init(self, name, memory, version, callback) 340 if name: 341 self._file = open(self.name, 'rb') --> 342 self._read() 343 344 else:
/usr/local/lib/python3.6/site-packages/asammdf/mdf_v4.py in _read(self) 596 size=size, 597 ) --> 598 data = next(data) 599 600 if record_id_nr == 0:
StopIteration:
@raffsa this is obviously an error in the asammdf package, not in mdfreader. Please open an issue there.