Implementing Meta-Curvature

[sudarshan1994]

Right now Meta-Curvature seems to be the best performing MAML variant for various few-shot learning tasks. In this technique, they learn precondition matrices for the gradients in the inner loop. I feel it would be useful to have a general framework where we could have a transform called inner gradient transform, which can modulate the inner loop gradients. So in this framework Meta-Curvature would be a special case. I would be happy to contribute if possible.

[tristandeleu]

This would be a great contribution indeed! You can have a look at the gradient_update_parameters for inspiration. Having a general transform sounds like an excellent idea, I would also suggest adding some utility functions (as syntactic sugar) for some standard methods (MAML or Meta-Curvature).

[sudarshan1994]

Hey thanks for getting back, I went through the code base--its a super useful contribution, thank the whole team for this ! I think a class wrapping the gradient_update_parameters which takes in gradient transform as an argument in its init would be a neat way to implement meta-curvature, plus other gradient transformations could be seamlessly integrated into the system this way.

[sudarshan1994]

Hey, so we have an implementation of metacurvature using pytorch-meta and these are the results on miniImageNet: 52 % for 1-shot and 63.8 % for 5-shot. The meta-curvature paper implements a couple of regularization tricks : cut-out and a data augmentation trick, plus they use a much wider CNN (128 filters), so the numbers we have cant be directly compared to the original paper. However this paper, has an implementation of meta-curvature with the standard CNN (32 filters) and does not apply any regularization tricks: Our 5-shot result on miniImageNet matches here (unfortunately they dont report 1-shot results :/)

[qbouniot]

Hi ! Could you share your implementation of Meta-Curvature ? I would be really interested in trying it out. Thank you !

[sudarshan1994]

Sure I ll clean up the code, give me a couple of weeks

[mfischer-ucl]

Hi @sudarshan1994 , any updates on this? I'd be interested in your implementation of Meta-Curvature as well! Thank you :)