Xiaole Zhu issues

Results 6 issues of


                                            Xiaole Zhu

pretrain什么开放啊，一年啦

What's the prompt for the caption task?

Thank you for your excellent work! I want to know the prompt that you train the caption task. Thank you for your reply at soon.

ValueError: Image features and image tokens do not match: tokens: 9888, features 9887

训练的时候出现了这个问题，图像的特征和token数量对不上，请问是什么原因，跑了两次都卡到应该是同一条数据上报这个错误，请问呢有遇到这种情况嘛？或者有什么解决方案嘛？

model_type = self._reverse_config_mapping[key.name] KeyError: 'Qwen2RMConfig'

### System Info File "/anaconda/envs/openrlhf/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 767, in __getitem__ model_type = self._reverse_config_mapping[key.__name__] KeyError: 'Qwen2RMConfig' transformers version is 4.51.3 ### Who can help? _No response_ ### Information - [ ] The...

bug

关于路由查询链接个数为8的权重？

作者您好，很有兴读到了这么好的文章，请问能分享一下路由链接查询为8的权重吗？想用一下其中的路由查询链接的emb的权重，我们跳过了一阶段，直接二阶段开始训练8token随即初始化之后发现效果比用4token的权重效果差一些，因为想请问是否可以分享一下8token的权重，谢谢！

model_type = self._reverse_config_mapping[key.name] KeyError: 'Qwen2RMConfig'

File "/data/LIMR/openrlhf/models/model.py", line 75, in get_llm_for_sequence_regression base_class = AutoModel._model_mapping[type(config)] File "/data/anaconda/envs/openrlhf/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 771, in __getitem__ model_type = self._reverse_config_mapping[key.__name__] KeyError: 'Qwen2RMConfig'