xtuner

xtuner copied to clipboard

LZHgrla

PoSE：https://arxiv.org/abs/2309.10400 https://github.com/OpenAccess-AI-Collective/axolotl/pull/1567 不需要全长度微调，无需修改 attention 好像可以用短上下文训练长上下文

2793145003

llava-surgery.py for phi3_mini to get gguf

1

I followed the instruction and got to `./iter_39620_hf` and `./iter_39620_llava`. I tried to convert them to gguf using the instrution [here ](https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README.md) but got into issues that I've seen [here](https://huggingface.co/xtuner/llava-llama-3-8b-v1_1/discussions/3)...

rezacopol

请问在基于单轮对话的微调时，想让模型在垂直领域回答的能力更强，对于数据集中的知识学习记忆效果更好在数据集处理以及参数上有什么tricks嘛，谢谢。

Toblame

multi image inputs supports for xtuner chat in llava-llama3?

1

- cmd: `xtuner chat LLM-Research/Meta-Llama-3-8B-Instruct \ --visual-encoder ./clip-vit-large-patch14-336 \ --llava ./LLM-Research/llava-llama-3-8b \ --prompt-template llama3_chat \ --image ./test001.png` - question: trained multimodal model can only input **one image** at one time...

ztfmars

模型是meta-Llama-3-8B 代码用例子中增加了FSDP的参数： model_wrapper_cfg=dict(type='MMFullyShardedDataParallel', cpu_offload=True,use_orig_params=True) # 指定 FSDPStrategy 并配置参数 size_based_auto_wrap_policy = partial( size_based_auto_wrap_policy, min_num_params=1e5) strategy = dict( type='FSDPStrategy', model_wrapper=dict(auto_wrap_policy=size_based_auto_wrap_policy)) 如果模型有quantization_config部分参数代码，则会报错 Must flatten tensors with uniform dtype but got torch.float32 and torch.bfloat16...

apachemycat

LZHgrla

xtuner训练 InternLM 1.8B模型报错 assert position_ids is not None and (position_ids.max() + 1) >= kv_seq_len

1

internlm2.py", line 154, in internlm2_attn_forward [rank0]: assert position_ids is not None and (position_ids.max() + 1) >= kv_seq_len [rank0]: Traceback (most recent call last): [rank0]: File "/usr/local/lib/python3.10/dist-packages/xtuner/tools/train.py", line 342, in [rank0]:...

apachemycat

llava-phi3 模型转换到llava模型报错。求助，怎么解决？

1

```python``` Traceback (most recent call last): File "/export/App/training_platform/PinoModel/xtuner/xtuner/configs/llava/phi3_mini_4k_v16/convert_xtuner_weights_to_llava.py", line 99, in main() File "/export/App/training_platform/PinoModel/xtuner/xtuner/configs/llava/phi3_mini_4k_v16/convert_xtuner_weights_to_llava.py", line 94, in main convert_to_llava(args.text_model_id, args.vision_model_id, File "/export/App/training_platform/PinoModel/xtuner/xtuner/configs/llava/phi3_mini_4k_v16/convert_xtuner_weights_to_llava.py", line 80, in convert_to_llava model.load_state_dict(state_dict, strict=True, assign=True) File...

awzhgw

convert_xtuner_weights_to_llava.py and convert_xtuner_weights_to_hf.py can support llava-next model?

2

i has use https://github.com/hhaAndroid/xtuner/tree/refactor_llava train llava_1.6_phi3_8B model . but ,it cannot convert to office llava model use convert_xtuner_weights_to_llava.py can you help me ? #641 @LZHgrla @hhaAndroid help me ? thanks

awzhgw