Poor results when fine-tuning a pretrained SVGG, compared to the standard VGG

[NicolaCST]

Issue type

• [ ] Bug Report
• [ ] Feature Request
• [x] Help wanted
• [x] Other

SpikingJelly version

'latest'

Description

Hi all, I'm trying to fine-tune a VGG19 pretrained on Imagenet on Flowers102. I'm freezing only the feature extractor, while for the classifier I use a custom one (is the same as the standard one, I just add another intermediate layer)

I'm having some hard times to complete a test: the ANN version performs really well (~70/80 acc). However the SNN requires an enormous amount of ram (Colab, I can barely reach the 16 batch size with T=10, multistep and cupy) and does not achieve good results (<5 acc)

Can you guys have experienced something similar? Do you have any idea/advice?

Thank you in advance

@fangwei123456

[fangwei123456]

Hi, the deep SNN is hard to train. You can try to train VGG19 with BN and check if it works. Note that the pre-trained weights from ANNs are not designed for SNNs.

the SNN requires an enormous amount of ram

The SNN is trained by BPTT, whose memory consumption is proportional to T * N. Thus, if the memory consumption of an ANN is X GB when batch size is N, then that of an SNN with the same structure will be at least T*X GB.