How to finetuning with lower memory fp16 version for p100 GPUs?

[xurongqiang]

For finetuning with lower memory fp16 version（for fp32 version , OOMs occur. ）, How should I modify the training.py script?

[keskarnitish]

This is quite delicate and doesn't quite seem to work out of the box. I'm going to need more time to look into this.

[xurongqiang]

This is quite delicate and doesn't quite seem to work out of the box. I'm going to need more time to look into this.

The background to this problem is that we have a large number of p100 machines, but they cannot run on the fp32 version. Thank you for your improvement.

[v1nc3nt27]

Is there some update on this matter? We have the same problem, unfortunately.

[itsuncheng]

Yes same here for us. Both huggingface and this repo seem to have the same OOM error when running on Google Colab free GPU like p100. Any fix or workaround yet?

[nastyatrvl]

The problem still persists, unfortunately. Fine-tuning doesn't really work with collab resources..