zserio icon indicating copy to clipboard operation
zserio copied to clipboard

Python support, but with better performance

Open MisterGC opened this issue 3 years ago • 3 comments

Some sample tests (no systematic testing done yet) indicated that the performance of apps using the zserio Python code (generated + runtime) is far worse than the one using the C++ code (C++ 10-100x faster). In the scope of an app this could mean <1 sec vs >10sec for parsing a set of (complex) zserio blobs.

Replacements for the current Python support that probably lead to significantly better performance:

  1. Use C++ + Python bindings (examples: https://github.com/jslee02/awesome-cpp-python-binding-generator) - this option could also have a positive effect regarding maintenance as C++ and Python support would share more code
  2. Use Cython

These (and other) options should be checked.

MisterGC avatar May 02 '22 08:05 MisterGC

Note that pybind11 is already extensively used in zswag ;)

josephbirkner avatar Jul 12 '22 08:07 josephbirkner

I think it can make sense to use some kind of python binding on top of the C++ generator, but it can still be only another "feature" provided by zserio. Python generator is based on pure Python (except the apsw) and it can be interesting for some users to use it as it is, without need to compile anything. Please also note that we never have time to optimize python generator at all.

Mi-La avatar Jul 15 '22 07:07 Mi-La

This issue is about only usage of C++ Python bindings. Meaning that we will try to bind C++ generator to Python and users will have to build C++ library for corresponding target by themselves.

Regarding pure Python optimization, new issue has been created: #466. Regarding usage of C++ implementation in Python runtime, new issue has been created: #469

mikir avatar Jan 20 '23 09:01 mikir