WangHeng

https://github.com/nerdyrodent/VQGAN-CLIP/blob/a6c8c487b89727d3c3440b8b3c406331c12275d6/generate.py#L726 Why calculate a mseloss of z and 0 and multiply it by a function on init_weight? Is it some kind of regular term?

stage2 VTG loss wrong ?

1

inputs_embeds = torch.cat([bos_embeds, img_embeds, to_regress_embeds], dim=1) you just concat all text emdeds with img_embeds? But BLIP2 concat half text emdeds with img_embeds. This is from blip2: "For encoder-decoder-based LLMs, we...

About the compute resources

2

Thanks for making your work public! Want to know how many computing resources were used for training and retrieval when you train the GPT-125M model?

can you release the code of making pseudo queries?

[BUG] <title>multi-image微调

1

### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? - [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions ### 该问题是否在FAQ中有解答？ | Is there an...