marie-ai icon indicating copy to clipboard operation
marie-ai copied to clipboard

Add support for Nvidia Multi-process Service (MPS)

Open gregbugaj opened this issue 1 year ago • 0 comments

Describe the feature Add support for CUDA-MPS to take advantage of new optimizations.

Requesting:

  • [ ] Setup scripts (Installation / Ansible / Docker )
  • [ ] Refactor exiting GPU/CPU from initialize_device_settings to device independent mode
  • [ ] Update models to extend GPU/CPU support for GPU/CPU/MPS/XPU

MPS Documentation

gregbugaj avatar Jan 18 '24 16:01 gregbugaj