是否支持批量推理?

[Zhouziyi828]

看到了支持单个图片调用模型进行推理,是否支持批量的推理?

[LDLINGLINGLING]

可以使用LMdeploy,代码如下 from lmdeploy import pipeline, TurbomindEngineConfig from lmdeploy.vl import load_image

可将MiniCPM-Llama3-V 2.5换成本地路径

#session_len=2048 代表上下文长度 #tp=8 代表使用显卡数，必须是2**n，比如1，2，4，8 pipe = pipeline('MiniCPM-Llama3-V 2.5', backend_config=TurbomindEngineConfig(session_len=2048,tp=8),)

图片的url或者本地路径

image_urls=[ "/root/ld/ld_project/MiniCPM-V/assets/minicpmv2-cases.png", "/root/ld/ld_project/MiniCPM-V/assets/llavabench_compare_phi3.png", "/root/ld/ld_project/MiniCPM-V/assets/MiniCPM-Llama3-V-2.5-peformance.png", "/root/ld/ld_project/MiniCPM-V/assets/zhihu.webp", "/root/ld/ld_project/MiniCPM-V/assets/thunlp.png" ] prompts = [('describe this image', load_image(img_url)) for img_url in image_urls] response = pipe(prompts) print([i.text for i in response])

[Zhouziyi828]

我用这个代码会出现: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. [WARNING] gemm_config.in is not found; using default GEMM algo
[WARNING] gemm_config.in is not found; using default GEMM algo Segmentation fault (core dumped) 然后没有任何输出

[whyiug]

最新的minicpmv-llama3模型已经可以batch inference了

这段代码是报错的。用的最新的model commit 3b6aeff3850ce9d5087751911e4771c78004b2b3

[rueing]

MiniCPM-V 请问该项目支持图片以url输入吗？

[1223243]

请问一下，有在开源微信社区里的吗

[LDLINGLINGLING]