DeepSpeed-MII icon indicating copy to clipboard operation
DeepSpeed-MII copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Is there a way for mii to not occupy all the available gpu memory

Open flexwang opened this issue 8 months ago • 0 comments

When I run examples in DeepSpeedExamples to start a server, it occupy all the GPU memory at the beginning. Is it possible to config the max gpu memory that it should occupy? Also what is the recommended way to host two models in the same container?

flexwang avatar Nov 22 '23 00:11 flexwang