Discrepancy between code and paper

[nshervt]

Good afternoon!

I have a question regarding your OML implementation. Based on the paper, parameters W are updated in the adaptation (inner) loop (Alg. 2, L. 9), while $\theta$ is updated in the meta-optimization step (Alg. 2, L. 12). However, in your code, both W and $\theta$ are passed to the meta-optimizer:

https://github.com/khurramjaved96/mrcl/blob/2855a6b7e820f171432981b58c49664fcdbf00ed/model/meta_learner.py#L189

which later updates these parameters in the meta step

https://github.com/khurramjaved96/mrcl/blob/2855a6b7e820f171432981b58c49664fcdbf00ed/model/meta_learner.py#L391-L396

Hence, to reflect the code, line 12 in Algorithm 2 should be modified as:

Question:

Should W, in fact, be updated in the meta-optimization loop? Or should we simply use W = W_k to update W based on the adaptation step and only pass $\theta$ to the meta-optimizer?

Thanks!