pyopy
pyopy copied to clipboard
Group computations to do more in octave land with less number of calls
Most of pyopy is geared towards this. Ideally we would:
- transfer each piece of data once into octave land
- group all operators into a single call (matlab for/parfor/cellfun)
- store efficiently in matlab land the outputs (meaning a struct of arrays instead of an array of structs)
- get back at once (or in batches to tame memory consumption) the outputs
There are two complications on matlab code generation for this task:
- [ ] Control and propagate errors
- [ ] The indetermined number of outputs from a call to certain operators, that not only depend on the curry but also on the time-series itself (the most simple example are features that are computed for certain lengths of the time series).
The first task, though, would be to properly measure the overhead of both method call and many-struct retrieval. It was really dominating computation with the initial versions of the bindings but now, after combining pymatbridge for calls and oct2py+ramdisk for data transfers, things seems much better in that respect. So it might be better to just let things as they are, simple and fairly robust, than saving an irrelevant amount of time.