gpt-fast icon indicating copy to clipboard operation
gpt-fast copied to clipboard

Published 20 hours ago • verified publisher icon pytorch-labs

Tensor Parallel Inside notebook

Open nivibilla opened this issue 9 months ago • 3 comments

Hi,

Im trying to get an example working with Ray on Databricks. Essentially having multiple replicas of the model. Is it possible to load a model with tensor parallelism inside a notebook?

Thanks

nivibilla avatar Apr 29 '24 19:04 nivibilla