cython-codegen icon indicating copy to clipboard operation
cython-codegen copied to clipboard

some experiment to use ctypes codegen for cython

This is a small package to generate cython code from header file: instead of having to write all the function declarations and structures by yourself, xml2cython can use gccxml output files to generate all the declarations automatically.

DISCLAIMER: I DON'T HAVE TIME TO SPEND ON THIS, AND YOU SHOULD CONSIDER IT UNSUPPORTED AT THAT POINT. A BETTER SOLUTION TO THE PROBLEM SOLVED BY CYTHON_CODEGEN WOULD BE SOMETHING BASED ON CLANG.

Dependencies

  • python
  • gccxml (recent build >= 0.9 from CVS)
  • codegenlib from ctypeslib

Usage

Say you have a header foo.h you want to wrap with cython. First, you need to generate an xml file from gccxml. You can use the h2xml script from ctypelibs:

    h2xml -I. foo.h -o foo.xml

You can then generate the cython file with xml2cython:

    xml2cython.py -l 'foo' foo.h foo.xml

This tells xml2cython to generate a cython file from foo.xml, originating from foo.h by gccxml, and to only pull items whose location match the string foo.

Filters

By default, xml2cython pull out every function in the xml file, and every 'dependency' (necessary to generate correct cython function declarations). This is likely to generate invalid cython code because of compilers intrisincs and so on, so you should filter the items pulled out from the xml file.

That's why I implemented of couple of naive filters. For example, say you are only interested in declarations coming from one header file, like AudioHardware.h (a header file for CoreAudio framework on Mac OS X). You can then use:

    xml2cython -l 'AudioHardware.h' CoreAudio/AudioHardware.h AudioHardware.xml

Only functions, enumerations and typedefs coming from this file will be pulled out. You should be able to use regex (as understood by python regex enging) for the filter string. Programmatically, it is easy to filter depending on the type of the item (function, typedef, etc...) but I have not found an easy way to present this to the user, so you will need to do it by yourself: look at the cycodegenlib.cycodegen.classify and xml2cython.py generate_main functions to see how basic filtering works.

Location

By default, xml2cython pull out every function in the xml file, and every 'dependency' (necessary to generate correct cython function declarations). This is likely to generate invalid cython code because of compilers intrisincs and so on, so you should filter the items pulled out from the xml file.

Location is a gccxml 'concept' related to the origin of each item in the parsed tree built by gccxml. Location refers to both file and line location for each C item (structure, typedef, function declaration, etc...). Simply using the -l option of xml2cython on the name of the header file hence will only pull functions declared in that file. You can also use a regex, which will be matched against the file location of each item.

If you need more control, you will likely need to do it by yourself.

Caveat

Limitations

Many. xml2cython is nothing more than an hack to avoid writing by hand cython files to wrap large API. Since I know nothing about compilers and parsing, it is likely that the implementation makes you laugh, too.

Many C declarations are not supported by cython (for example complex numbers) and no check is made that xml2cython generates valid cython code. Any header file which crashes xml2cython is a bug, but a non-buildable cython file may not be.

Another problem is that the query system is limited, meaning the generated files are quickly very big, hence slow to compile with cython. I am adding a few filters capabilities (to filter on function names, file origin, etc...), but it is unlikely to get fancy.

That being said, xml2cython is useful :) It can generate valid code for non trivial header (I am using it successfully to wrap alsa - a C api with > 1000 functions, hundred of typedefs and structures as well as CoreAudio on Mac OS X).

C vs C++

Unfortunately, gccxml only parses C++ files. So if your header is not C++ compatible, it will not work. No C++ concept is supported by xml2cython, most of them are removed from the AST by xml2cython to avoid conflict with cython (which does not support C++ constructs either).

LICENSE

The code is under MIT license (See LICENSE). I don't care about the code I wrote myself (all the code in this package except gccxmlparser.py): I can license it do any license you want if you request it. The only reason why I used this is because I use some code from ctypeslib, under the MIT. If the license is not acceptable, you should rewrite the parts which depend on ctypeslib (mostly gccxmlparse as well as type descriptors)/