AlexJJJChen

I don't know why it always showed this error, anyone could help me? thank you so much! (textgen) j@ai1:~/llama/text-generation-webui$ python server.py --model LLaMA-7B --load-in-8bit --no-stream Loading LLaMA-7B... Traceback (most recent...

Traceback (most recent call last): File "/Users/xxx/Documents/OpenCastKit/predict.py", line 10, in backbone_model.load('./weights/fourcastnet/backbone.pt') File "/opt/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1269, in __getattr__ raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'AFNONet' object has no attribute...

**Describe the bug** CUDA_VISIBLE_DEVICES=0,1,2,3 swift export \ --ckpt_dir finetune_output/checkpoint-478 --load_dataset_config true \ --quant_method awq --quant_bits 4 \ --merge_lora true \ Traceback (most recent call last): File "/home/jianc/miniconda3/envs/benchmark-llm/lib/python3.10/site-packages/swift/cli/export.py", line 5, in...

finetune 好的模型用不了

1

from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration import torch from PIL import Image import requests from modelscope import snapshot_download from transformers import AutoModelForCausalLM, AutoTokenizer from peft import AutoPeftModelForCausalLM device_count = torch.cuda.device_count() if...

Multimodal model & neftune_noise_alpha is not compatible. Dimension out of range.

2

CUDA_VISIBLE_DEVICES=0,1,2,3 \ swift sft \ --neftune_noise_alpha "5" \ --model_id_or_path "AI-ModelScope/llava-v1.6-mistral-7b" \ --template_type "llava-mistral-instruct" \ --custom_train_dataset_path xxx.json \ --custom_val_dataset_path xxx.json \ --dataset_test_ratio "0.2" \ --save_steps "50" \ --lora_target_modules q_proj k_proj v_proj...

webui点击开始训练之后一晚上没有出任何结果

1

**Describe the bug** What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) **Your hardware and system info** Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等) **Additional context**...

使用DDP运行时显存不够，但是使用Model Parallel时可以正常finetune，耗时很大

6

nproc_per_node=4 CUDA_VISIBLE_DEVICES=0,1,2,3 \ NPROC_PER_NODE=$nproc_per_node \ swift sft \ --model_id_or_path "AI-ModelScope/llava-v1.6-mistral-7b" \ --template_type "llava-mistral-instruct" \ --custom_train_dataset_path train_swift.json \ --custom_val_dataset_path test_swift.json \ --dataset_test_ratio "0.15" \ --save_steps "20" \ --lora_target_modules q_proj k_proj v_proj...

Traceback (most recent call last): File "/home/jianc/benchmark/finetune_s2s_model.py", line 232, in model = get_peft_model(base_model, lora_config) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/mapping.py", line 133, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File "/home/jianc/anaconda3/envs/benchmark-llm/lib/python3.10/site-packages/peft/peft_model.py", line 1043, in __init__...

我想将模型保存到本地，怎么才能保存。同时怎么调用本地的模型？

infer 怎么指定输出文件夹？

2

infer 怎么指定输出文件夹？