ChatGLM3训练时报错TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

[AILWQ]

运行脚本如下：

deepspeed --include localhost:6 --master_port 520 train.py \
                --train_path data/spo_0.json \
                --model_name_or_path ChatGLM3-6B \
                --per_device_train_batch_size 1 \
                --max_len 1560 \
                --max_src_len 1024 \
                --learning_rate 1e-4 \
                --weight_decay 0.1 \
                --num_train_epochs 2 \
                --gradient_accumulation_steps 4 \
                --warmup_ratio 0.1 \
                --mode glm3 \
                --lora_dim 16 \
                --lora_alpha 64 \
                --lora_dropout 0.1 \
                --lora_module_name "query_key_value,dense_h_to_4h,dense_4h_to_h,dense" \
                --seed 1234 \
                --ds_file ds_zero2_no_offload.json \
                --gradient_checkpointing \
                --show_loss_step 10 \
                --output_dir ./output-glm3

报错如下：

[2024-01-07 00:44:07,189] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-07 00:44:10,270] [INFO] [launch.py:145:main] WORLD INFO DICT: {'localhost': [6]}
[2024-01-07 00:44:10,271] [INFO] [launch.py:151:main] nnodes=1, num_local_procs=1, node_rank=0
[2024-01-07 00:44:10,271] [INFO] [launch.py:162:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0]})
[2024-01-07 00:44:10,271] [INFO] [launch.py:163:main] dist_world_size=1
[2024-01-07 00:44:10,271] [INFO] [launch.py:165:main] Setting CUDA_VISIBLE_DEVICES=6
[2024-01-07 00:44:12,382] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
  File "/home/wangyingli/liwenqiang/ChatGLM-Finetuning/train.py", line 22, in <module>
    from utils import print_trainable_parameters, print_rank_0, to_device, set_random_seed, save_model
  File "/home/wangyingli/liwenqiang/ChatGLM-Finetuning/utils.py", line 15, in <module>
    from transformers import set_seed
  File "/home/wangyingli/.local/lib/python3.9/site-packages/transformers/__init__.py", line 26, in <module>
    from . import dependency_versions_check
  File "/home/wangyingli/.local/lib/python3.9/site-packages/transformers/dependency_versions_check.py", line 16, in <module>
    from .utils.versions import require_version, require_version_core
  File "/home/wangyingli/.local/lib/python3.9/site-packages/transformers/utils/__init__.py", line 18, in <module>
    from huggingface_hub import get_full_repo_name  # for backward compatibility
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/home/wangyingli/.local/lib/python3.9/site-packages/huggingface_hub/__init__.py", line 357, in __getattr__
    submod = importlib.import_module(submod_path)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/wangyingli/.local/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 49, in <module>
    from huggingface_hub.utils import (
  File "/home/wangyingli/.local/lib/python3.9/site-packages/huggingface_hub/utils/__init__.py", line 45, in <module>
    from ._headers import build_hf_headers, get_token_to_send, LocalTokenNotFoundError
  File "/home/wangyingli/.local/lib/python3.9/site-packages/huggingface_hub/utils/_headers.py", line 20, in <module>
    from ._runtime import (
  File "/home/wangyingli/.local/lib/python3.9/site-packages/huggingface_hub/utils/_runtime.py", line 62, in <module>
    _package_versions[candidate_name] = importlib.metadata.version(name)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/importlib/metadata.py", line 569, in version
    return distribution(distribution_name).version
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/importlib/metadata.py", line 542, in distribution
    return Distribution.from_name(distribution_name)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/importlib/metadata.py", line 192, in from_name
    dist = next(iter(dists), None)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 739, in <genexpr>
    path.search(prepared) for path in map(FastPath, paths)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 631, in search
    return self.lookup(self.mtime).search(name)
  File "/home/wangyingli/.conda/envs/competition/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 636, in mtime
    return os.stat(self.root).st_mtime
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

麻烦聪哥看下这是什么问题导致的，弄一晚上了没解决