Ability to clone neural nets instead of all the weights/biases tensors manually

[creikey]

Right now have to manually clone all the weights, bug prone

[mratsim]

What code did you use for cloning?

[creikey]

@mratsim Like this:

    intoBrain.gru.w3s0 = ctx.variable(fromBrain.gru.w3s0.value.clone())
    intoBrain.gru.w3sN = ctx.variable(fromBrain.gru.w3sN.value.clone())
    intoBrain.gru.u3s = ctx.variable(fromBrain.gru.u3s.value.clone())
    intoBrain.gru.bW3s = ctx.variable(fromBrain.gru.bW3s.value.clone())
    intoBrain.gru.bU3s = ctx.variable(fromBrain.gru.bU3s.value.clone())

    intoBrain.fc1.weight = ctx.variable(fromBrain.fc1.weight.value.clone())
    intoBrain.fc1.bias = ctx.variable(fromBrain.fc1.bias.value.clone())
    intoBrain.fc2.weight = ctx.variable(fromBrain.fc2.weight.value.clone())
    intoBrain.fc2.bias = ctx.variable(fromBrain.fc2.bias.value.clone())

Where the brain is like this:

  CreatureBrain[T] = object
    gru: GRULayer[T]
    memory: Variable[Tensor[T]]
    fc1: Linear[T]
    fc2: Linear[T]

[mratsim]

As a workaround in the meantime, you can use fields iterator here: https://nim-lang.org/docs/iterators.html#fields.i%2CS%2CT

proc copyWeights[T](intoBrain: var T, fromBrain: T) =
  for dst, src in fields(intoBrain, fromBrain):
    when dst is Variable:
      dst = ctx.variable(src.value.clone())
    else:
      dst.copyWeights(src)