Yifan Du

https://github.com/dandelin/ViLT/blob/762fd3975c180db6fc88f577cf39549983fa373a/vilt/modules/vision_transformer.py#L636 Thanks for your awesome work! I have a question about pad_choice. When choosing pads to pad the sequence, you sample invalid patches with `replacement=True`. I wonder if this will...

No model.zero_grad()

1

Thanks for your awesome work! I noticed you comment the mode.zero_grad() in https://github.com/j-min/VL-T5/blob/18699e2b1d5b4559f76c88a48cdec7176d356c34/VL-T5/src/caption.py#L225 So how could the model update its parameters?

您好，非常感谢您的开源代码！ 请问中文分词的数据需要处理成什么样子呢？从SIGHAN上下载的是txt格式的，但是代码里需要的是json格式，我也没有在README里找到样例数据。您可以提供一下处理好之后的数据，或者数据预处理的代码吗？ 十分感谢～

生成任务的数据

1

您好，官网上下载的数据格式和模型需要的输入不一致，麻烦您可以也发一下代码能接收的csl和adgen的数据吗？十分感谢～ _Originally posted by @fdugzc in https://github.com/fastnlp/CPT/issues/13#issuecomment-1006350187_

Thanks for your awesome work! Could you please offer the requirement.txt or environment.yml for convenient installation?

AssertError in prompt tuning

4

https://github.com/OFA-Sys/OFA/blob/3982ed1eec4230e99bcb70f34c0fead05f6643d3/models/ofa/unify_multihead_attention.py#L318 When I execute prompt tuning on the image caption task, the AssertError arises. The length of prompt is 100, so `key_padding_mask.size(1)=117`, and `k.size(1)=17`. I'm not sure how to fix...

Thanks for your awesome work and detailed README! However, when I perform preprocessing with process_data.py, the output directory and file `wiki_one_article_per_line.txt` is empty. I think the input file of process_data.py...

Pre-training code of BLIP-2

8

Thanks for your awesome work in BLIP-2, it displays surprising abilities when conjoining LLM and image encoder! Do you plan to release the code to pre-train such a model? We...

Error in beam-search multinomial sampling

1

According to the transformer in Huggingface, beam-search multinomial sampling can be implemented by setting `num_beams>1` and `do_sample=True`. However, this is not supported in LAVIS. If I set `num_beams=4, num_return_sequences=4` and...

A question about training cost

4

Thanks for your awesome work! Would you mind mentioning the training cost of LLaVA (including both the GPU hours and cost of API)? Thanks in advance :)