BlueBlueFF

Results 17 comments of BlueBlueFF

Bug: The dimension is not match when prepare decorder attention mask.

same question

[Usage] tokenization mismatch when finetuning v1.5-7b

> I have fixed the issue, You just need to make sure the inputs and targets properly masked. Can you share your tokenizer settings?

[Question]When I use llava v1.6-34b, an error occurs -RuntimeError: CUDA error: peer mapping resources exhausted

Same Questions

请问InternVL_1.5_26B的LoRA微调版本什么时候能出来？

> > 亲测1.2版本的lora微调可以用在1.5版本上，但需要做以下更改 ![image](https://private-user-images.githubusercontent.com/144629782/333088717-93c73fb3-3fd4-4e82-86f5-535efe96a639.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTY0NTA4MjUsIm5iZiI6MTcxNjQ1MDUyNSwicGF0aCI6Ii8xNDQ2Mjk3ODIvMzMzMDg4NzE3LTkzYzczZmIzLTNmZDQtNGU4Mi04NmY1LTUzNWVmZTk2YTYzOS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNTIzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDUyM1QwNzQ4NDVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lMzE4NDJiNjdmZmUwZjUyNzUwNTc3ZWQ4ODBlYjRiOTdlMTZhZTFkMDMyZGUxYjcxYjJmNGZkNjlkMTBmODM1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.W5fkZlGHGPTP6vOTpYacFPHC2xJ9s9hytZFVONf0vjU) > > 同时merge lora的权重时可以借鉴swift库 merge lora这里有更具体一些的代码吗 swift封装太深 没太看明白。。

Ablation study on using just single path encoder?

> The model you mentioned with a single visual path is exactly LLaVA-1.5. We have conduct extensive comparisons in Tab 1 of our paper. Check out our [paper](https://arxiv.org/pdf/2403.03003.pdf). ![image](https://private-user-images.githubusercontent.com/38316576/310429347-cac412ea-e0bb-454e-9d43-310fd958e136.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTAxMjI3MzUsIm5iZiI6MTcxMDEyMjQzNSwicGF0aCI6Ii8zODMxNjU3Ni8zMTA0MjkzNDctY2FjNDEyZWEtZTBiYi00NTRlLTlkNDMtMzEwZmQ5NThlMTM2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAzMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMzExVDAyMDAzNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWNjMDI1ZDA5ZmFkZTdiNjdhODNjYjc0MThlMDhjM2FiNmE3NWU2NGM3ZjU3YzFiZWQ1NDUyNWM0MDZkNGQwZWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.-LdB6Mm0Lp1wQ3cp21triMSLgNZS1EJ_hFCRIB_998E) In...

Ablation study on using just single path encoder?

> > The model you mentioned with a single visual path is exactly LLaVA-1.5. We have conduct extensive comparisons in Tab 1 of our paper. Check out our [paper](https://arxiv.org/pdf/2403.03003.pdf). ![image](https://private-user-images.githubusercontent.com/38316576/310429347-cac412ea-e0bb-454e-9d43-310fd958e136.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTAxMjI3MzUsIm5iZiI6MTcxMDEyMjQzNSwicGF0aCI6Ii8zODMxNjU3Ni8zMTA0MjkzNDctY2FjNDEyZWEtZTBiYi00NTRlLTlkNDMtMzEwZmQ5NThlMTM2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAzMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMzExVDAyMDAzNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWNjMDI1ZDA5ZmFkZTdiNjdhODNjYjc0MThlMDhjM2FiNmE3NWU2NGM3ZjU3YzFiZWQ1NDUyNWM0MDZkNGQwZWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.-LdB6Mm0Lp1wQ3cp21triMSLgNZS1EJ_hFCRIB_998E)...

如何混合图文数据和纯文本数据训练？

> 不需要，支持纯文本训练 数据上改如何组织呢？ 比如BatchSize=8，对于纯文本的Pixel，这时候如何和图文的pixel做concat？