D-SCRIPT
D-SCRIPT copied to clipboard
Add multi-GPU support for prediction (and embedding?)
A single process splits candidate interactions to the number of allocated GPUs, and all are able to access the same loaded embeddings in memory
- currently can't run more than a certain number of different jobs in parallel, because each load all the embeddings into their memory, which fills up system memory quickly
- for large numbers of predictions, running in serial on a single GPU is a waste of time and system resources
- fix: cmd line allocates which GPUs are able to be used, and sub processes run on each one, after embeddings are loaded into a shared memory