PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

from_pretrained加载本地模型报错

Open fly2 opened this issue 3 years ago • 1 comments

  • 版本、环境信息 1)PaddleNLP和PaddlePaddle版本:PaddleNLP 2.3.5,PaddlePaddle2.3.1 2)系统环境:Linux:Debian GNU/Linux 11,python:3.8.12
  • 复现信息:如为报错,请给出复现环境、复现步骤

下载ernie-3.0-medium-zh模型到本地后加载报错, 文件本地存放路径:

os.listdir('/model/ernie-3.0-medium-zh/')
['ernie_3.0_medium_zh_vocab.txt', 'model_config.json', 'special_tokens_map.json', 'tokenizer_config.json', 'vocab.txt', 'ernie_3.0_medium_zh.pdparams']

错误信息:

from paddlenlp.transformers import *
ernie = AutoModel.from_pretrained('/model/ernie-3.0-medium-zh/')
[2022-08-17 06:07:26,217] [    INFO] - We are using <class 'paddlenlp.transformers.ernie.modeling.ErnieModel'> to load '/model/ernie-3.0-medium-zh/'.
[2022-08-17 06:07:26,218] [    INFO] - Downloading /model/ernie-3.0-medium-zh/model_state.pdparams and saved to /model/ernie-3.0-medium-zh/
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/paddlenlp/transformers/auto/modeling.py", line 390, in from_pretrained
    return cls._from_pretrained(pretrained_model_name_or_path, task,
  File "/usr/local/lib/python3.8/site-packages/paddlenlp/transformers/auto/modeling.py", line 251, in _from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/lib/python3.8/site-packages/paddlenlp/transformers/model_utils.py", line 272, in from_pretrained
    resolved_resource_files[file_id] = get_path_from_url(
  File "/usr/local/lib/python3.8/site-packages/paddlenlp/utils/downloader.py", line 156, in get_path_from_url
    assert is_url(url), "downloading from {} not a url".format(url)
AssertionError: downloading from /model/ernie-3.0-medium-zh/model_state.pdparams not a url


fly2 avatar Aug 17 '22 06:08 fly2

抱歉,这里有些不太完善。您那边是直接使用的下载缓存的目录是吗,可以先save_pretrained保存一次然后使用保存的目录来加载,save过程中有些转换适配的处理,之前是考虑from_pretrained加载本地目录的话需要是保存适配过的,我们也看看完善下

guoshengCS avatar Aug 18 '22 02:08 guoshengCS

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

github-actions[bot] avatar Dec 08 '22 02:12 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。

github-actions[bot] avatar Dec 22 '22 16:12 github-actions[bot]

需要先save_pretrained,直接从缓存目录中下载下来的信息不齐全,无法直接使用

songsurging avatar Jan 02 '24 09:01 songsurging