Arraymancer icon indicating copy to clipboard operation
Arraymancer copied to clipboard

Support nested TrainableLayer

Open mratsim opened this issue 5 years ago • 1 comments

This is necessary to build NN models by block

The current TrainableLayer concept only check the first level fields for Variable:

https://github.com/mratsim/Arraymancer/blob/90263efabdf706d7a74c1d847b7449495d67c124/src/nn_dsl/dsl_types.nim#L61-L79

Furthermore TrainableLayers should be ironed out and moved into the nn module.

It might be fused with what is called "Model" in optimizers:

https://github.com/mratsim/Arraymancer/blob/90263efabdf706d7a74c1d847b7449495d67c124/src/nn/optimizers/optimizers.nim#L43-L55

mratsim avatar Sep 29 '18 14:09 mratsim

The change is probably easy from:

 TrainableLayer*[TT] = concept layer 
   block: 
     var trainable: false 
     for field in fields(layer): 
       trainable = trainable or (field is Variable[TT]) 
     trainable 

to

 TrainableLayer*[TT] = concept layer 
   block: 
     var trainable: false 
     for field in fields(layer): 
       trainable = trainable or (field is Variable[TT]) or (field is TrainableLayer[TT])
     trainable 

hopefully recursive concepts don't break.

mratsim avatar Oct 25 '18 15:10 mratsim