ChatGLM3 TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding

System Info / 系統信息

torch==2.4.0 transformers==4.45.0

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

[X] The official example scripts / 官方的示例脚本
[ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

Step1

升级到transformers到4.45.0

Step2

运行如下代码：


from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True).half().cuda()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)
response, history = model.chat(tokenizer, "晚上睡不着应该怎么办", history=history)
print(response)

Expected behavior / 期待表现

应该需要更新HF仓库中的:ChatGLMTokenizer._pad 的参数

Sep 26 '24 06:09 jeejeelee

@zRzRzRzRzRzRzR 辛苦花点时间看下这个问题，glm4也存在类似问题，非常感谢~

Sep 27 '24 03:09 jeejeelee

import torch from datasets import Dataset from diffusers import FluxPipeline, KolorsPipeline from PIL import Image # 导入PIL库用于保存图片

from yijian_community.defense import InternVL2ImageDefense, ThuCoaiShieldLM from yijian_community.model import HFTxt2ImgInfer

def txt2img_zh_usage_example(txt2img_zh_model="/mnt/public/lqz/prompt_attack/Kolors-diffusers"): txt2img_zh = HFTxt2ImgInfer( model_path=txt2img_zh_model, pipe=KolorsPipeline, variant="fp16", )

# 生成一张图片
text_prompt = "今天天气很好。"
img = txt2img_zh.infer_data(
    data=text_prompt, guidance_scale=5.0, num_inference_steps=50
)


# 保存图片到指定路径
output_path = "test.jpg"
img.save(output_path)  # 保存图片

print(f"图片已保存到 {output_path}")

txt2img_zh_usage_example()

kolor文本好像用的是chaglm3，现在也会报错： TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side'

环境： Package Version

accelerate 0.34.2 aiohappyeyeballs 2.4.0 aiohttp 3.10.5 aioprometheus 23.12.0 aiosignal 1.3.1 annotated-types 0.7.0 anyio 4.5.0 async-timeout 4.0.3 attrs 24.2.0 bert-score 0.3.13 certifi 2024.8.30 charset-normalizer 3.3.2 click 8.1.7 cloudpickle 3.0.0 colorama 0.4.6 contourpy 1.3.0 cycler 0.12.1 datasets 3.0.0 diffusers 0.30.3 dill 0.3.8 diskcache 5.6.3 distro 1.9.0 einops 0.8.0 exceptiongroup 1.2.2 fastapi 0.115.0 filelock 3.16.1 fonttools 4.53.1 frozenlist 1.4.1 fsspec 2024.6.1 gguf 0.10.0 h11 0.14.0 httpcore 1.0.5 httptools 0.6.1 httpx 0.27.2 huggingface-hub 0.25.0 idna 3.10 importlib_metadata 8.5.0 iniconfig 2.0.0 interegular 0.3.3 jieba 0.42.1 Jinja2 3.1.4 jiter 0.5.0 joblib 1.4.2 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 kiwisolver 1.4.7 lark 1.2.2 llvmlite 0.43.0 lm-format-enforcer 0.10.6 lxml 5.3.0 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.9.2 mdurl 0.1.2 mistral_common 1.4.3 mpmath 1.3.0 msgpack 1.1.0 msgspec 0.18.6 multidict 6.1.0 multiprocess 0.70.16 nest-asyncio 1.6.0 networkx 3.3 ninja 1.11.1.1 numba 0.60.0 numpy 1.26.4 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-ml-py 12.560.30 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.6.68 nvidia-nvtx-cu12 12.1.105 openai 1.50.0 orjson 3.10.7 outlines 0.0.46 packaging 24.1 pandas 2.2.2 partial-json-parser 0.2.1.1.post4 pillow 10.4.0 pip 24.2 pluggy 1.5.0 portalocker 2.10.1 prometheus_client 0.21.0 prometheus-fastapi-instrumentator 7.0.0 protobuf 5.28.2 psutil 6.0.0 py-cpuinfo 9.0.0 pyairports 2.1.1 pyarrow 17.0.0 pycountry 24.6.1 pydantic 2.9.2 pydantic_core 2.23.4 Pygments 2.18.0 pyparsing 3.1.4 pytest 8.3.3 python-dateutil 2.9.0.post0 python-dotenv 1.0.1 pytz 2024.2 PyYAML 6.0.2 pyzmq 26.2.0 quantile-python 1.1 ray 2.36.0 referencing 0.35.1 regex 2024.9.11 requests 2.32.3 rich 13.8.1 rouge-metric 1.0.1 rpds-py 0.20.0 sacrebleu 2.4.3 safetensors 0.4.5 scikit-learn 1.5.2 scipy 1.14.1 sentence-transformers 3.1.1 sentencepiece 0.2.0 setuptools 59.6.0 six 1.16.0 sniffio 1.3.1 starlette 0.38.5 sympy 1.13.3 tabulate 0.9.0 threadpoolctl 3.5.0 tiktoken 0.7.0 timm 1.0.9 tokenizers 0.20.0 tomli 2.0.1 torch 2.4.0 torchvision 0.19.0 tqdm 4.66.5 transformers 4.45.1 triton 3.0.0 typing_extensions 4.12.2 tzdata 2024.1 urllib3 2.2.3 uvicorn 0.30.6 uvloop 0.20.0 vllm 0.6.2 watchfiles 0.24.0 websockets 13.0.1 xformers 0.0.27.post2 xxhash 3.5.0 yarl 1.11.1 yijian-community 0.1.4 zipp 3.20.2

Sep 27 '24 15:09 lilqz66

chatglm3-6b-32k报错

Sep 29 '24 04:09 LeiyanGithub

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题

Sep 29 '24 07:09 bjr-yql

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

Sep 30 '24 14:09 LeiyanGithub

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

4.43.0 应该可以解决这个问题。 PS：建议遇到类似问题的，可以去GLM4的repo提issue，可能会响应更快点(GLM4也存在类似的问题)

Sep 30 '24 15:09 jeejeelee

modelscope上说了是chatglm不支持最新版本

Oct 01 '24 01:10 huolongguo1O

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

4.43.0 应该可以解决这个问题。 PS：建议遇到类似问题的，可以去GLM4的repo提issue，可能会响应更快点(GLM4也存在类似的问题)

之前的问题解决了，貌似出现了新的问题，vllm不支持这个版本！

Oct 01 '24 07:10 LeiyanGithub

已经收到这个问题，预计节后投入人力解决

Oct 01 '24 11:10 zRzRzRzRzRzRzR

Hi,

I ran into the same error in the context of llama.cpp trying to convert https://huggingface.co/THUDM/glm-4-9b-chat to GGUF.

Error

TypeError: ChatGLM4Tokenizer._pad() got an unexpected keyword argument 'padding_side'

Workaround

Downgrade the transformers package to version 4.44.2:

pip uninstall transformers -y; pip install transformers==4.44.2

It seems as if this bug is introduced with the transformers release v4.45.0.

I hope this helps someone :wink:

Oct 02 '24 09:10 countzero

降低版本到4.40.2，可以了 pip uninstall transformers -y; pip install transformers==4.40.2

Oct 08 '24 19:10 wuhanzhuyang

Can Transformers 4.45.0 run without errors now? If not, any plan to address this?

Oct 09 '24 07:10 Orenji-Tangerine

换成4.40会出现新的错误，怎么办？ ModuleNotFoundError: No module named 'transformers.models.phi3'

Oct 09 '24 12:10 BLANKBLK

换成4.40会出现新的错误，怎么办？ ModuleNotFoundError: No module named 'transformers.models.phi3'

换4.42 或移除phi3相关代码

Oct 09 '24 14:10 huolongguo1O

Hi guys I have the same issue recently, even though i mainly use FLUX, I recently switched to KOLORS, which is literally MJV6.1 killer with lower CFG and samplers, But I have this issue of the GLM thing, I tried everything and its not working! pls help!

Oct 12 '24 06:10 ghost

We have received this question and plan to invest manpower to solve it after the holiday.

please!!!!!

Oct 12 '24 06:10 ghost

Hi guys I have the same issue recently, even though i mainly use FLUX, I recently switched to KOLORS, which is literally MJV6.1 killer with lower CFG and samplers, But I have this issue of the GLM thing, I tried everything and its not working! pls help!

Same as this issue?

Oct 12 '24 10:10 huolongguo1O

Hi,

I ran into the same error in the context of llama.cpp trying to convert https://huggingface.co/THUDM/glm-4-9b-chat to GGUF.

Error
TypeError: ChatGLM4Tokenizer._pad() got an unexpected keyword argument 'padding_side'
Workaround

Downgrade the transformers package to version 4.44.2:
pip uninstall transformers -y; pip install transformers==4.44.2
It seems as if this bug is introduced with the transformers release v4.45.0.

I hope this helps someone 😉

unfortunately it didn't help bro! thank you though

Oct 13 '24 03:10 ghost

可以参考这个解决方案 https://huggingface.co/THUDM/LongWriter-glm4-9b/commit/778b5712634889f5123d6c463ca383bc6dd5c621#d2h-069285

Oct 17 '24 08:10 Foluolaaa

+1

Nov 11 '24 06:11 ztsweet

I tried to apply this fix https://huggingface.co/THUDM/chatglm3-6b/discussions/64 it works on my side

Nov 14 '24 08:11 eaidova

@zRzRzRzRzRzRzR glm2-6b也存在TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side'這個問題，已嘗試調降transformer版本，但依舊會出現TypeError。能麻煩您再去看一下這個問題嗎?麻煩您了，謝謝。