youngjae-you

@PawelPeczek-Roboflow I think the version of the `transformers` package in dockerfile-gpu related to **_roboflow inference_** is the offending version. I'd like to lower it and give it a try, but...

@we0091234 Then can I understand that there is no compatibility problem with the previous code?

@dusty-nv Thank you. The image resolution is 336, and you didn't use token compression, right?

> @YoungjaeDev it depends on the model, but IIRC VILA 1.5 is using SigLIP 384x384 vision encoder, and the 3B model compresses it down to ~192 image tokens using the...

@Yogesh914 Is it possible to take real-time camera input and reference it? How long does it take to reference?

[sample_finetune.zip](https://github.com/user-attachments/files/16937575/sample_finetune.zip) The cuda:0 was being used elsewhere, and I'm trying to train and infer with cuda:1 at the same time, but the logs on the reference side are holding cuda:0.

Anyone else relate to this?

```python File "/home/user/res/lmdeploy/lmdeploy/serve/async_engine.py", line 521, in getprompt_input prompt = chat_template.messages2prompt(prompt, File "/home/user/res/lmdeploy/lmdeploy/model.py", line 223, in messages2prompt if len(messages) and messages[0]['role'] != 'system': TypeError: string indices must be integers ``` The...

@radna0 Use `TurbomindEngineConfig` ```python from lmdeploy import pipeline, ChatTemplateConfig, TurbomindEngineConfig ... backend_config = TurbomindEngineConfig(tp=2) model = pipeline('Lin-Chen/ShareCaptioner', backend_config=backend_config, chat_template_config=ChatTemplateConfig(model_name='internlm-xcomposer2-4khd'), log_level='INFO') ```

> Thank you, Do you have the slide captioning batch inference working for the ShareCaptioner-Video model? I'm looking at the code right now, trying to set up inference on a...