open-simulation-interface
open-simulation-interface copied to clipboard
OpenSimulationInterface
Support more protobuf versions in GitHub workflow
Reference to a related issue in the repository
#765
Add a description
Fix pipeline behavior with different protobuf versions
Take this checklist as orientation for yourself, if this PR is ready for the Change Control Board:
- [x] My suggestion follows the style and contributors guidelines.
- [x] I have taken care about the message documentation and the fields and enums documentation.
- [x] I have done the DCO signoff.
- [x] My changes generate no errors when passing CI tests.
- [x] I have successfully implemented and tested my fix/feature locally.
- [x] Appropriate reviewer(s) are assigned.
If you can’t check all of them, please explain why. If all boxes are checked or commented and you have achieved at least one positive review, you can assign the label ReadyForCCBReview!
Protobuf 2.6.1 does not work out of the box, since it does not have '-all' in the link/package: https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7609941012
If '-all' is changed to '', it will download abseil, but cannot install it, since protobuf 2.6.1 has autotools (configure). So the if condition distinguishing between autotools and cmake is not correct: https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7609969004
Also, between 3.0.0 and 3.5.0 there is no package for all environments.
With my changes, protobuf 2.6.1 can be installed. But now installing python fails: https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7610124644
Protobuf versions >= 3.5.1 seem to work now.
Proto3 versions below don't work because of a compiler error. See for example https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7613993384/job/20735224384
Protobuf version 2.6.1 fails during the OSI setup. See https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7614005171/job/20735265068
Proto3 versions below don't work because of a compiler error. See for example https://github.com/OpenSimulationInterface/open-simulation-interface/actions/runs/7613993384/job/20735224384
I fixed this by setting CXXFLAGS=-std=c++11 for configure.
Now, only protobuf 2.6.1 does not work.
I tried again with protobuf 2.6.1. The problem occurs during building the python files form the .proto files.
When running protoc --python_out osi3/ osi3/osi_version.proto the following error occurs:
google/protobuf/descriptor.proto: File not found.
In osi_version.proto, google/protobuf/descriptor.proto is included. However I checked, and it is located in /usr/include and /usr/local/include. So I have no idea, why it is not found and why this only is an issue with protobuf 2.6.1 and not the other versions.
Has anyone ever used the combination osi - protobuf 2.6.1 - python? @pmai do you have any idea what might be the issue?
Another sanity check: Do older versions of protobuffer also work with e.g. C++17? I would guess yes
Another sanity check: Do older versions of protobuffer also work with e.g. C++17? I would guess yes
For the older versions I had to explicitly set a CXXFlag to c++11. But this works for all versions.
I now managed to get the setup.py to work with protobuf 2.6.1. However, now a unit test fails. Apparently Protobuf 2.6.1 does not support Python 3...
My recommendation: Let's set the minimum proto version to 3.0.0, as it already says in the documentation anyways.
I now managed to get the setup.py to work with protobuf 2.6.1. However, now a unit test fails. Apparently Protobuf 2.6.1 does not support Python 3...
My recommendation: Let's set the minimum proto version to 3.0.0, as it already says in the documentation anyways.
At least officially protobuf states Python 3 support since 2.6.0... The error logs look more like there are changes in the location of the bundled proto include?
I now managed to get the setup.py to work with protobuf 2.6.1. However, now a unit test fails. Apparently Protobuf 2.6.1 does not support Python 3... My recommendation: Let's set the minimum proto version to 3.0.0, as it already says in the documentation anyways.
At least officially protobuf states Python 3 support since 2.6.0... The error logs look more like there are changes in the location of the bundled proto include?
What about this here? https://github.com/protocolbuffers/protobuf/issues/882
Considerations:
- IF protobuffer min. version is set to >3.0.0 THEN we should switch to proto3 syntax
- Remove proto 2to3 converter
- Remove second build step
- check flatc compatibility with proto3 syntax
@ClemensLinnhoff and @pmai this PR must be closed and we have to check if there is anything missing. See also my questions above. I think we are still inconsistent as the docs still mention proto version >3.0.0 and then my questions with the proto syntax arise!
Python is now completely decoupled from the protobuf version. So the changes to setup.py are superseded. What is the status on proto 2 support for C++? Then we might need the changes. However, only the CI pipeline has minor changes.