textgen icon indicating copy to clipboard operation
textgen copied to clipboard

Published 20 hours ago • verified publisher icon shibing624

多卡部署

Open ZTurboX opened this issue 2 years ago • 3 comments

想请问一下，用fastapi多卡部署生成式模型，怎样在不同卡上起服务呢

ZTurboX avatar Jun 06 '23 02:06 ZTurboX

每个卡起一个server，再做负载均衡。

shibing624 avatar Jun 06 '23 03:06 shibing624

每个卡起一个server，再做负载均衡。

请问有案例吗

ZTurboX avatar Jun 06 '23 14:06 ZTurboX

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.(由于长期不活动，机器人自动关闭此问题，如果需要欢迎提问)

stale[bot] avatar Dec 27 '23 07:12 stale[bot]