Baichuan-13B icon indicating copy to clipboard operation
Baichuan-13B copied to clipboard

Published 20 hours ago • verified publisher icon baichuan-inc

Baichuan-13B-Chat可以批量调用吗？

Open FutureForMe opened this issue 1 year ago • 5 comments

你好，请问基于Baichuan-13B-Chat模型可以使用batch进行批量调用吗？

FutureForMe avatar Aug 18 '23 03:08 FutureForMe

same question

zldeng avatar Aug 18 '23 03:08 zldeng

你好，请问基于Baichuan-13B-Chat模型可以使用batch进行批量调用吗？

https://github.com/yanzihan1/Baichuan-batch-inference 简单demo

yanzihan1 avatar Aug 22 '23 07:08 yanzihan1

可以直接使用 FastChat，兼容 openai 的 api

forcemeter avatar Sep 04 '23 07:09 forcemeter

可以直接使用 FastChat，兼容 openai 的 api

baichuan2 can be loaded with fastchat, but the performance is very different from that of normal deployment. I don't know why the performance is lost.

ye7love7 avatar Sep 12 '23 14:09 ye7love7

你好，请问基于Baichuan-13B-Chat模型可以使用batch进行批量调用吗？

自己改了一个批量生成的示例 https://github.com/MrInouye/baichuan-13b-chat-batch/tree/main

MrInouye avatar Nov 17 '23 07:11 MrInouye