[question] How `clear` works?

[al6x]

Hi, can you please explain how the clear works? The docs says

clear - If set to true, will clear the network after every activation. This is useful for training LSTM's, more importantly for timeseries prediction. Default: false

I'ts not very clear what's the every activation means. Does it mean that it will be cleared after every example? If so - how would it be trained to predict - let's say string parsing.

Let's say we want to parse strings with LSTM:

'two'   => 2
'one'   => 1
'seven' => 7

Every single example would be converted into group - a sequence of multple onehots.

In my understanding clear should be called not for every activation (every onehot) but for every group (mutliple onehots). See example below:

(0 and 1 are random, just to demonstrate)

// first example 'two' => 2
clear
[0, 0, 1, 0] => [0, 0] // t => nothing
[0, 1, 0, 0] => [0, 0] // w => nothing
[1, 0, 0, 0] => [0, 1] // o => 2

// second example 'one' => 1
clear
[0, 0, 1, 0] => [0, 0] // o => nothing
[0, 1, 0, 0] => [0, 0] // n => nothing
[1, 0, 0, 0] => [0, 1] // e => 1

But it seems like what the docs are saying is the following

// first example 'two' => 2
clear
[0, 0, 1, 0] => [0, 0] // t => nothing
clear
[0, 1, 0, 0] => [0, 0] // w => nothing
clear
[1, 0, 0, 0] => [0, 1] // o => 2

// second example 'one' => 1
clear
[0, 0, 1, 0] => [0, 0] // o => nothing
clear
[0, 1, 0, 0] => [0, 0] // n => nothing
clear
[1, 0, 0, 0] => [0, 1] // e => 1

So, how does the clear works? Is it per onehot or per group of onehots?

[al6x]

Actually I figured it out - you need to set clear: false and add one more onehot for termination signal (the neural network should be smart enough to figure out to clear itself when see it), like that:

// first example 'two' => 2
[0, 0, 1, 0, 0] => [0, 0] // t => nothing
[0, 1, 0, 0, 0] => [0, 0] // w => nothing
[1, 0, 0, 0, 0] => [0, 1] // o => 2
[0, 0, 0, 0, 1] => [0, 0] // special onehot signalling to do clear

// second example 'one' => 1
[0, 0, 1, 0, 0] => [0, 0] // o => nothing
[0, 1, 0, 0, 0] => [0, 0] // n => nothing
[1, 0, 0, 0, 0] => [0, 1] // e => 1
[0, 0, 0, 0, 1] => [0, 0] // special onehot signalling to do clear