Bend icon indicating copy to clipboard operation
Bend copied to clipboard

Published 20 hours ago • verified publisher icon HigherOrderCO

Request: Multi GPU parallelism

Open Jainam2130 opened this issue 9 months ago • 2 comments

Does this natively support parallelism accross gpus? Also feature request: natively perform flash attention please

Jainam2130 avatar May 18 '24 08:05 Jainam2130

It doesn't support working across multi gpus yet. First we would require a 64bit implementation of HVM to be able to use all resources and then a significant change to the cuda runtime to be able to use multiple GPUS. An interesting idea to pursue in the not-so-far future, but not our immediate priority.

developedby avatar May 18 '24 09:05 developedby

Also does this support parallelism across the whole system (eg across CPU + multiple GPUs)?

dxvid-pts avatar May 18 '24 16:05 dxvid-pts