mPLUG-Owl

mPLUG-Owl copied to clipboard

how to realize multi-image correlation in vqa task?

4

In vqa task, I want to input two images and ask a question about the two images,how to realize it?

fansticOne

other downstream tasks available? Like Visual Reasoning, requires the model to predict whether a sentence describes a pair of images

1

fansticOne

对图像进行坐标检测，生成的bbox是resize成正方形之后的值吗？

5

对图像进行坐标检测，生成的bbox是resize成正方形之后的值吗？ 因为实例代码是将图像resize成正方形了，我跑出来结果有点偏差，所以想问下你们是正方形吗？

zhaop-l

I'm looking for a checkpoint for an mPLUG model suitable for processing multilingual (mainly Chinese) videos. Does such a checkpoint already exist? If not, are there other model checkpoints suitable...

abeldafa

unable to load model

3

Hello, I am trying to load the model using the demo but I get ``` tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, load_8bit=False, load_4bit=False, device="cuda") File "mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/builder.py", line 110,...

segalinc

Encountered flash_attn_2_cuda error while running finetune_lora.sh

1

While running finetune_lora.sh on colab, i encountered the following issue. ``` Traceback (most recent call last): File "/content/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/train/llama_flash_attn_monkey_patch.py", line 10, in from flash_attn.flash_attn_interface import flash_attn_unpadded_qkvpacked_func File "/usr/local/lib/python3.10/dist-packages/flash_attn/__init__.py", line 3, in...

pradipto111

多卡设备运行 **[Quick Start Code](https://github.com/X-PLUG/mPLUG-Owl/tree/main/mPLUG-Owl2#quick-start-code)** 时报错，修改 device 设置单卡可以。但是单卡显存太小（T4卡），只能 8bit 或者 4bit ，请问怎么可以调整成多卡且默认float16运行吗？

wanghanyang123

cur_input_embeds = torch.cat([cur_input_embeds_1, cur_image_features[0:0], cur_input_embeds_2], dim=0),其中cur_image_features[0:0]表示这是一个没有维度的向量，图像的特征并没有真正加进去

1

mPLUG-Owl2中的代码错误

hangzeli05

NameError: name 'repeat_kv' is not defined

1

Hello, When I run the Quick Start demo, I encounter the following problem： File "/home/wxz/LLM/mPLUG_Owl2/mplug_owl2/model/modeling_llama2.py", line 139, in forward key_states = repeat_kv(key_states, self.num_key_value_groups) NameError: name 'repeat_kv' is not defined Is...

nullnameno

mPLUG-Owl2，finetune，训练卡住没有输出

5

参考LLAVA的方式构建自己的数据集，数据量14w，按照官方给的方式进行finetune，发现进入trainer.train()之后就hang住了，strout和strerr也没有信息，没有报错，请问是哪里不对吗 strout中的输出 ```python [2023-12-04 15:00:43,681] [WARNING] [partition_parameters.py:823:_post_init_method] param `cls_token` in MplugOwlVisionEmbeddings not on GPU so was not broadcasted from rank 0 [2023-12-04 15:00:43,747] [WARNING] [partition_parameters.py:823:_post_init_method] param `position_embedding` in MplugOwlVisionEmbeddings not...

LianghuiGuo