Lai Wei

[Question]: How to write the stream_complete function for my custom LLM (used for Gradio)?

1

### Question Validation - [X] I have searched both the documentation and discord for an answer. ### Question ``` class HuatuoLLM(CustomLLM): dummy_response: str = "My response" @property def metadata(self) ->...

Qwen1.5-7B-base模型推理速度比Qwen1.5-7B-chat模型速度慢很多

3

如题。我使用的huggingface格式的推理。请问是什么原因呢？

Can I fine-tune minigpt-v2 on 4090?

3

I set the batch_size to 2 for training on 4*4090 gpus. But it still shows that 'Cuda out of memory'.

Open-source?

3

Any plan for open-sourced models?

I found the link of pre-trained ckpt and sft ckpt the same.

MiniGPT-4 and LLaVA evaluation

1

Hi! I'm a fan of your work. Can you please provide more details about how to do eval for MiniGPT-4 and LLaVA on various datasets? Thanks a lot!

Can we use in-context multimodal data for finetuning?

6

Thanks for your great work! However, it seems that we can only use data that contains one image for SFT. Can we use in-context multimodal data (i.e., containing multiple images)...

Does LLaVA-Next support in-context(few-shot) inference?

4

Thanks for your work! Can I input multi-images and multi-instructions for few-shot inference?

Thanks for your work. Do you follow the initialization in RWKV Github Repo?

Thanks for your great work! Can I input multi-images and multi-instructions for few-shot inference?