Can unlimiformer be trained on mutiple gpus?

[Muxv]

Is there ant option I can modify to support mutiple gpus training?

[urialon]

Hi @Muxv , Thank you for your interest in our work!

It is currently not fully supported. I tried to run with multiple GPUs once, but I got errors that the "memory" tensors are not on the same GPU as the query tensors. We would love to adopt contributions.

Best, Uri