deepnet icon indicating copy to clipboard operation
deepnet copied to clipboard
verified publisher icon nitishsrivastava

Tied weights and TransposedCUDAMatrix

Open rdaniel opened this issue 12 years ago • 3 comments

Hi,

I'm trying to modify the autoencoder model_layer1.pbtxt file to tie the weights from the hidden layer to the output layer to be the transpose of those from the input layer to the hidden layer. When I run this I get the error:

AttributeError: 'TransposedCUDAMatrix' object has no attribute 'shape'

Any ideas how to fix this? I tried making the TransposedCUDAMatrix be derived from CUDAMatrix instead of object, which fixed that problem but then died when trying to use the .T operator.

Thanks, Ron

rdaniel avatar May 02 '13 22:05 rdaniel

Daniel,

  • Could please tell me where you add this modification ? I am also interested in this. We could add a special config parameter like "tied_transpose_to:" or something like.

cbalint13 avatar Dec 30 '13 07:12 cbalint13

I've manged it. Will come back a bit later with a patch proposal, and with results over "ae" example. Will post it over issue #48.

cbalint13 avatar Jan 01 '14 15:01 cbalint13

  • It is nonsense to tie a transposed bias/weight (e.g in case of autoencoder) to some other layer in the very learning process. However I manage it in the code in case of feed_forward_net, but not really usefull at all.

cbalint13 avatar Mar 04 '14 13:03 cbalint13