AgentNet icon indicating copy to clipboard operation
AgentNet copied to clipboard

better weights management for memory layers

Open justheuristic opened this issue 7 years ago • 0 comments

Right now recurrent memory cells have clumsy weight management.

  • manual weight initialization during layer creation https://github.com/yandexdataschool/AgentNet/blob/master/agentnet/memory/rnn.py#L260-L266
  • inside there's more than one layer so one can't simply get lstm.W_this_to_that without e.g. calling lasagne.layers.get_all_params(lstm)
  • more to be added

Some ideas on how to make that better

  • reimplement all rnn.py as whole layers like GRUMemoryLayer (otherwise deprecate it)
    • extra work for farther development
    • may introduce bugs
  • add some MacroLayer that wraps any lasagne network
    • simple to implement in current lasagne (see f0k comment here)
    • yet another abstraction
  • add some dict notation for weight initialization to make it more human readable

in case someone actually reads this, pls share your ideas. I'd be astonished to know that you exist.

justheuristic avatar Nov 07 '16 23:11 justheuristic