sicara
How to run with Relation network?
Can you provide an example of how the code can be run with relation networks?
Hi! You can follow the Quickstart. Just replace PrototypicalNetworks by RelationNetworks in step 5 and you should be all set!
@ebennequin I replaced PrototypicalNetworks by RelationNetworks. But I get error "Illegal backbone for Relation Networks. Expected output for an image is a 3-dim tensor of shape (n_channels, width, height)". Can you tell me why output of backbone should be tensor of shape (n channel, width, height). Looking forward to hearing from you. Thanks <3
RelationNetworks differ from PrototypicalNetworks in that it doesn't simply compute euclidean distances in the feature space, but instead put a "relation module" on top of it. So that we don't lose spatial information, the relation module doesn't take flattened feature vectors as input, but instead takes "feature maps" of shape (n_channels, width, height). So for RelationNetworks we expect a backbone which output has this same shape.
I explicitely raised an error to avoid more "shadow error" at runtime, but if it's not clear I probably should rethink that.
@ebennequin, how would relation network be trained with classical training method? For prototypical networks, it would be okay to just train the backbone model with classical training. For the relation network, how would the relation module be trained? Do you suggest to train the backbone with classical training method first, and then train the relation module with episodic training? Thank you very much for your guidance!
In the original paper, the backbone and relation module are fully trained with episodic training. As I understand it, more recent methods using a backbone and an additional module
- first pre-train the backbone with classical training
- then train the whole thing (backbone + module) with episodic training
I guess you can freeze the backbone during the second step. I don't see any reason why it couldn't work. If you run these experiments, I'm very interested in the results!
Got that, I also think it would work with the steps you mentioned! Thank you!
hi i try to run relation network but it says illegal backbone, can i know which part of code i need to edit for it to run?
You probably have a backbone that outputs a feature vector (1-dimensional) for each image. Relation Networks use feature maps (3-dimensional). If your backbone ends with a pooling layer, removing that layer should do the trick.
i still could not manage to run but i am using your backbone tutorial in classical training, could u specify which part of the resnet code to edit, my error is 'runtime error: only batches of spatial targets supported (3D tensors) but got targets of size [128]' after i have commented out the pooling layer