MiniCPM-V
MiniCPM-V copied to clipboard
OpenBMB
我的M3芯片本地运行MiniCPM-Llama3-V-2_5-int4得到了报错Using `bitsandbytes` 8-bit quantization requires Accelerate
是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?
- [X] 我已经搜索过FAQ | I have searched FAQ
当前行为 | Current Behavior
我已经在conda环境中安装了所需要的依赖，在本地mac上执行PYTORCH_ENABLE_MPS_FALLBACK=1 python xxx.py命令的时候，得到了报错。
期望行为 | Expected Behavior
得到输出
复现方法 | Steps To Reproduce
执行PYTORCH_ENABLE_MPS_FALLBACK=1 python xxx.py
运行环境 | Environment
- OS: Macos M3 sonoma14.3
- Python: 3.10
- Transformers:4.40.0
- PyTorch: 2.1.2
- CUDA : None，使用的MPS
其他pip安装的环境
Package Version
------------------------- ------------
accelerate 0.30.1
addict 2.4.0
aiofiles 23.2.1
altair 5.3.0
annotated-types 0.7.0
anyio 4.4.0
attrs 23.2.0
bitsandbytes 0.42.0
blis 0.7.11
Brotli 1.1.0
catalogue 2.0.10
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
cloudpathlib 0.16.0
colorama 0.4.6
confection 0.1.5
contourpy 1.2.1
cycler 0.12.1
cymem 2.0.8
editdistance 0.6.2
einops 0.7.0
et-xmlfile 1.1.0
exceptiongroup 1.2.1
fairscale 0.4.0
fastapi 0.110.3
ffmpy 0.3.2
filelock 3.14.0
fonttools 4.53.0
fsspec 2024.5.0
gmpy2 2.1.5
gradio 4.26.0
gradio_client 0.15.1
h11 0.14.0
httpcore 1.0.5
httpx 0.27.0
huggingface_hub 0.23.2
idna 3.7
importlib_resources 6.4.0
Jinja2 3.1.4
joblib 1.4.2
jsonlines 4.0.0
jsonschema 4.22.0
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
langcodes 3.4.0
language_data 1.2.0
lxml 5.2.2
marisa-trie 1.1.1
markdown-it-py 3.0.0
markdown2 2.4.10
MarkupSafe 2.1.5
matplotlib 3.7.4
mdurl 0.1.2
more-itertools 10.1.0
mpmath 1.3.0
murmurhash 1.0.10
networkx 3.3
nltk 3.8.1
numpy 1.24.4
opencv-python-headless 4.5.5.64
openpyxl 3.1.2
orjson 3.10.3
packaging 23.2
pandas 2.2.2
Pillow 10.1.0
pip 24.0
portalocker 2.8.2
preshed 3.0.9
protobuf 4.25.0
psutil 5.9.8
pydantic 2.7.3
pydantic_core 2.18.4
pydub 0.25.1
Pygments 2.18.0
pyparsing 3.1.2
PySocks 1.7.1
python-dateutil 2.9.0.post0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.1
referencing 0.35.1
regex 2024.5.15
requests 2.32.3
rich 13.7.1
rpds-py 0.18.1
ruff 0.4.7
sacrebleu 2.3.2
safetensors 0.4.3
scipy 1.13.1
seaborn 0.13.0
semantic-version 2.10.0
sentencepiece 0.1.99
setuptools 70.0.0
shellingham 1.5.4
shortuuid 1.0.11
six 1.16.0
smart-open 6.4.0
sniffio 1.3.1
socksio 1.0.0
spacy 3.7.2
spacy-legacy 3.0.12
spacy-loggers 1.0.5
srsly 2.4.8
starlette 0.37.2
sympy 1.12.1
tabulate 0.9.0
thinc 8.2.3
timm 0.9.10
tokenizers 0.19.1
tomlkit 0.12.0
toolz 0.12.1
torch 2.1.2
torchvision 0.16.2
tqdm 4.66.1
transformers 4.40.0
typer 0.9.4
typing_extensions 4.12.1
tzdata 2024.1
urllib3 2.2.1
uvicorn 0.24.0.post1
wasabi 1.1.3
weasel 0.3.4
websockets 11.0.3
wheel 0.43.0
备注 | Anything else?
日志：
lib'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
warn(
Unused kwargs: ['_load_in_4bit', '_load_in_8bit', 'quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.
Traceback (most recent call last):
File "/Users/xxx/miniCPM/local_main.py", line 7, in <module>
model = AutoModel.from_pretrained('/Users/gaobo60/aiModel/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True)
File "/Users/xxx/miniCPM/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
return model_class.from_pretrained(
File "/Users/xxxxx/miniCPM/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3165, in from_pretrained
hf_quantizer.validate_environment(
File "/Users/xxxx/miniCPM/lib/python3.10/site-packages/transformers/quantizers/quantizer_bnb_4bit.py", line 62, in validate_environment
raise ImportError(
ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`
由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。
请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError(
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。
请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError( ImportError: Using
bitsandbytes8-bit quantization requires Accelerate:pip install accelerateand the latest version of bitsandbytes:pip install -i https://pypi.org/simple/ bitsandbytes
bitsandbytes 目前仅支持cuda设备
非常感谢
在 2024-06-17 20:37:17，"Hongji Zhu" @.***> 写道：
由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。
请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
bitsandbytes 目前仅支持cuda设备
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
I'm trying to fine tune "openbmb/MiniCPM-Llama3-V-2_5-int4" with custom dataset. but getting error
"[2024-07-16 11:22:47,459] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-dev package with apt
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1
[WARNING] using untested triton version (2.1.0), only 1.0.0 is known to be compatible
/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/transformers/training_args.py:1474: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead
warnings.warn(
[2024-07-16 11:23:00,796] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-07-16 11:23:00,796] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
Unused kwargs: ['load_in_4bit', 'load_in_8bit', 'quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.
low_cpu_mem_usage was None, now set to True since model is quantized.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:35<00:00, 17.78s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Currently using LoRA for fine-tuning the MiniCPM-V model.
Traceback (most recent call last):
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune.py", line 328, in
train()
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune.py", line 274, in train
model.base_model.vpm.requires_grad (True)
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2440, in requires_grad
p.requires_grad_(requires_grad)
RuntimeError: only Tensors of floating point dtype can require gradients
[2024-07-16 11:24:03,027] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 129003) of binary: /mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/python3
Traceback (most recent call last):
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/torchrun", line 8, in
sys.exit(main())
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
run(args)
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
elastic_launch(
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
finetune.py FAILED
Failures: <NO_OTHER_FAILURES>
Root Cause (first observed failure): [0]: time : 2024-07-16_11:24:02 host : AlticeLab. rank : 0 (local_rank: 0) exitcode : 1 (pid: 129003) error_file: <N/A> traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html"
the code is "#!/bin/bash
GPUS_PER_NODE=1 NNODES=1 NODE_RANK=0 MASTER_ADDR=localhost MASTER_PORT=6001
MODEL="openbmb/MiniCPM-Llama3-V-2_5-int4" # or openbmb/MiniCPM-V-2
ATTENTION: specify the path to your training data, which should be a json file consisting of a list of conversations.
See the section for finetuning in README for more information.
DATA="/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/vl_finetune_data.json" EVAL_DATA="/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune-evaluate_data.json" LLM_TYPE="minicpm" # if use openbmb/MiniCPM-V-2, please set LLM_TYPE=minicpm
DISTRIBUTED_ARGS="
--nproc_per_node $GPUS_PER_NODE
--nnodes $NNODES
--node_rank $NODE_RANK
--master_addr $MASTER_ADDR
--master_port $MASTER_PORT
"
/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/torchrun $DISTRIBUTED_ARGS finetune.py
--model_name_or_path $MODEL
--llm_type $LLM_TYPE
--data_path $DATA
--eval_data_path $EVAL_DATA
--remove_unused_columns false
--label_names "labels"
--prediction_loss_only false
--bf16 false
--bf16_full_eval false
--fp16 true
--fp16_full_eval true
--do_train
--do_eval
--tune_vision true
--tune_llm false
--use_lora true
--lora_target_modules "llm..*layers.\d+.self_attn.(q_proj|k_proj)"
--model_max_length 2048
--max_slice_nums 9
--max_steps 10000
--eval_steps 1000
--output_dir output/output_minicpmv2_lora
--logging_dir output/output_minicpmv2_lora
--logging_strategy "steps"
--per_device_train_batch_size 2
--per_device_eval_batch_size 1
--gradient_accumulation_steps 1
--evaluation_strategy "steps"
--save_strategy "steps"
--save_steps 1000
--save_total_limit 10
--learning_rate 1e-6
--weight_decay 0.1
--adam_beta2 0.95
--warmup_ratio 0.01
--lr_scheduler_type "cosine"
--logging_steps 1
--gradient_checkpointing true
--deepspeed ds_config_zero2.json
--report_to "tensorboard" # wandb
"
