brainstorm

brainstorm copied to clipboard

double buffering

4

For double buffering to work properly, the new strategy of copying data to device in `provide_external_data` instead of the iterators requires some changes to how InputLayer works. This is a...

flukeskywalker

Retaining context across forward passes

5

We should have a `context_reset_rate` parameter (subject to renaming) in the trainer which is set by the `train` function. Using this, the context should be reset (cleared) if `current_update_nr` %...

flukeskywalker

Describing and serializing objects

16

From what I can see, the mechanism for generating descriptions and serializing networks etc. does not fully work yet. @Qwlouse, you were working on this. Any comments on what else...

flukeskywalker

All buffers have to be of the same dtype

3

Currently, all buffers (parameters, internals, gradients, ...) are assumed to have the dtype (typically either float or double). This is a bit restrictive: For example, in a max-pooling operation, one...

untom

Streams

14

Sooner or later, we should think about introducing CUDA streams for our GPU implementation. Side-Effect: Looking at the profiling outputs, across various example the most expensive call we make is...

untom

flukeskywalker

I think we should run some rudimentary profiling before the initial release, just to get rid of the worst performance offenders. On a related note, it'd be nice to have...

Qwlouse