CppADCodeGen

CppADCodeGen copied to clipboard

Hello,

I wanted to share a problem I ran into using this very well written and very useful library. I work using high dimensional models and my computational graphs tend to be large.

If the computational graph reaches a certain threshold, and the number of arrays and symbols exceed 2GB, we run into a relocation truncated to fit error. You can read more about this kind of error here https://stackoverflow.com/questions/6093547/what-do-r-x86-64-32s-and-r-x86-64-64-relocation-mean/33289761#33289761

This problem can in principle be addressed by linking with -mcmodel=large or mcmodel=medium flags. However in this case the library generated by CppADCodeGen requires linking to libc.a, which is built in 32 bit mode. Therefore the problem persists.

I believe that if the generated code was C++ with linkage to libgcc.a instead, the problem would be addressed. This in principle should require very little change since from my observations of the generated code all the mathematical operations are supported in C++ and the only difference should be changing the header to include <cmath> and of course renaming the files to use a cpp extension.

I would be happy to make the code changes, but I wanted to understand if that is in principle possible and whether the subsequent loading of the library after compilation would fail if the code was compiled with C++ mangling in the method signatures.

@joaoleal Are you still supporting CppADCodeGen ?

Hello,

Before starting to change/extend CppADCodeGen (which would still lead to extremely long compilation times) I would suggest looking for opportunities to reduce the size of the generated code for your model. This would solve your current problem and lead to significantly lower compilation times. If you have some part of your model with repeating equations (the larger the better) you can compile that section of the code and reuse it inside your model as an atomic model. For instance, see dynamic_atomic_nonlinear_outer.cpp/CppADCGDynamicAtomicTest.hpp.

It is also possible to change the "C" compiler and/or its flags. From the link you provided, it appears that using position-independent code would solve your problem. For this, you can create a dynamic library instead of a static library or create a static library but provide the option posIndepCode = true.

I hope this helps.

@bradbell support is typically limited to weekends ;)

I modified the generator to emit c++ code and this issue is resolved. There are cases when the graphs need to be big for high dimensional problems.