Speed up Decode and Encode timing

[xRowe]

I see there is a old thread on this topic, check if this topic can be carried on. #362

Is there any new solution to speed up decoding and encoding timing?

Online receving Messages by python-can and decoding: canmatrix v.s cantools

Example Test Code

def on_message_received(msg):
        start = time.perf_counter_ns()

        dbc_msg = _database.frame_by_id(msg.arbitration_id)
        
        # start = time.perf_counter_ns()
        decoded = dbc_msg.decode(msg.data)
        end = time.perf_counter_ns()
        
       timings.append(end-start)

canmatrix decode Mean +- std dev: 49.1 us +- 36.5 us (2207 values) Mean +- std dev: 97.4 us +- 182.5 us (12201 values) received_message + decode Mean +- std dev: 147.1 us +- 157.3 us (12201 values)

cantools decode Mean +- std dev: 30.0 us +- 29.9 us (2200 values) Mean +- std dev: 34.8 us +- 35.9 us (12201 values) received_message + decode Mean +- std dev: 64.9 us +- 57.7 us (12201 values)

After looking at cantools solution, after database is parsed, it will generate a _codec info which allowed to more quickly encode/decode a message, and generate two dicts which allow quick search message by name or id.

But canmatrix, everytime when receive a raw message, I need to found it in database by using for test in self.frames:, when call decode() method, it try to upack the data by bytes_to_bitstrings() and bitstring_to_signal_list(), which also using for loop.

I do not if bitsruct may help better, and refer to cantools

[ebroecker]

Hi @xRowe

thanks for doing the measurements!

It would be easy to implement some dictionary to allow quick search message by name or id. Thus I'm gonna implement such a hash-table which I think is a good idea.

Bitstruct on the other hand is (or at least was) not able to handle multiplexed messages. Multiplexer enables messages(=frames/pdus) to have different layouts depending on the content of some bits.

[xRowe]

Thanks for your reply.

There are commits along with implementing the DICTs already,

https://github.com/ebroecker/canmatrix/blob/e7ce2aa5e48d1d1bdb6226f35154e05c947dbf50/src/canmatrix/canmatrix.py#L1994-L2002 https://github.com/ebroecker/canmatrix/blob/e7ce2aa5e48d1d1bdb6226f35154e05c947dbf50/src/canmatrix/canmatrix.py#L2029-L2036

And I create/add the data here, but I think the way is not good enough, and not cover all the cases. https://github.com/ebroecker/canmatrix/blob/e7ce2aa5e48d1d1bdb6226f35154e05c947dbf50/src/canmatrix/canmatrix.py#L2075-L2089

Though my project is running these code, I did not run the benchmark then.

If you have better idea, please improve it.

Thanks, Rowe