numpyeigen icon indicating copy to clipboard operation
numpyeigen copied to clipboard

Add python scripts to call codegen stuff

Open fwilliams opened this issue 7 years ago • 2 comments

Let's have scripts that do what cmake does. This makes it easy to integrate numpyeigen into other build systems.

fwilliams avatar Jul 19 '18 18:07 fwilliams

+1 on this feature, as we are currently trying to integrate numpyeigen into a bazel-based build (do you have recommendations / experience on this end already?)

ph03 avatar Jun 24 '22 09:06 ph03

Hi, I don't have experience with Bazel unfortunately but would love to support build systems beyond CMake. I'd love help with this especially from someone who knows other build systems!

I can give you a rundown of what CMake currently does to codegen binding files and link them into a library:

Building your dependency happens in two steps:

  1. I am building a static library npe which contains typedefs, headers and dependencies needed to build your binding. Code to build this static library is here from line 172-178. Note the include(numpyEigenDependencies) at line 160. All this does is download pybind11 and Eigen so you can link them in. You can see the downloads in [this file](function(numpyeigen_download_pybind11). I'm depending on my fork of pybind11 which fixes some bugs and exposes some aspects of the NumPy API internally.

  2. We need to build a shared library which can be imported in Python. This is done by calling npe_add_module(<PYTHON_LIBRARY_NAME>, <YOUR_CPP_FILES>). This function is defined here at line 200. Foreach source file src.cpp you pass in, this function creates a custom command (triggered when that source file changes) which automatically calls python src/codegen_function.py which generates src.out.cpp in the build directory. codegen_function.py transpiles your binding into a raw C++ binding. The function then adds another custom command which calls codegen_module.py which generates a C++ file (<PYTHON_LIBRARY_NAME>.cpp) to export your functions as python bindings. Finally all the generated cpp files are added to a CMake target called <PYTHON_LIBRARY_NAME> and linked with the npe library defined in step 1. You can treat this target which you can treat as any normal build target.

So I guess to integrate with bazel you would want to do something similar and download dependencies (these are all header only so should be easy), then call codegen_function.py for each file and codegen_module.py to generate the module source code.

Hope this is helpful!

fwilliams avatar Jun 24 '22 23:06 fwilliams