aws-virtual-gpu-device-plugin
aws-virtual-gpu-device-plugin copied to clipboard
Support per_process_gpu_memory_fraction for pytorch and spacy models?
Setting per_process_gpu_memory_fraction seems to work for our tensorflow models, but doesn't seem to work for our pytorch and spacy models. So we consistently run into Out of memory errors since spacy, and pytorch models are using the same vgpus as our tensorflow models?
I know torch released this: https://pytorch.org/docs/stable/generated/torch.cuda.set_per_process_memory_fraction.html#torch.cuda.set_per_process_memory_fraction in 1.8.0, is there a way to use this flag with this plugin? Is there an equivalent with spacy models?