pyopy icon indicating copy to clipboard operation
pyopy copied to clipboard

Group computations to do more in octave land with less number of calls

Open sdvillal opened this issue 10 years ago • 0 comments

Most of pyopy is geared towards this. Ideally we would:

  • transfer each piece of data once into octave land
  • group all operators into a single call (matlab for/parfor/cellfun)
  • store efficiently in matlab land the outputs (meaning a struct of arrays instead of an array of structs)
  • get back at once (or in batches to tame memory consumption) the outputs

There are two complications on matlab code generation for this task:

  • [ ] Control and propagate errors
  • [ ] The indetermined number of outputs from a call to certain operators, that not only depend on the curry but also on the time-series itself (the most simple example are features that are computed for certain lengths of the time series).

The first task, though, would be to properly measure the overhead of both method call and many-struct retrieval. It was really dominating computation with the initial versions of the bindings but now, after combining pymatbridge for calls and oct2py+ramdisk for data transfers, things seems much better in that respect. So it might be better to just let things as they are, simple and fairly robust, than saving an irrelevant amount of time.

sdvillal avatar Oct 24 '14 13:10 sdvillal