`Invalid fd was supplied: -1` optimum.onnxruntime load onnx model failed

[L1-M1ng]

I used onnx package load my llm model directly successfully in my ubuntu x86 platform. model = onnx.load("Qwen_models/Qwen-7B-onnx/qwen_model.onnx") for node in model.graph.node: print(node.op_type) pass but failed with optimum，my code is as bellow： tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-7B", trust_remote_code=True, pad_token="<|endoftext|>") model = ORTModelForQuestionAnswering.from_pretrained( model_id="L1-m1ng/qwen7b-inf", #"Qwen_models/Qwen-7B-onnx/", trust_remote_code=True )

The complete error message is as bellow： Traceback (most recent call last): File "/home/mingli/projects/Qwen/run_ort_optimize.py", line 20, in model = ORTModelForQuestionAnswering.from_pretrained( File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/optimum/onnxruntime/modeling_ort.py", line 664, in from_pretrained return super().from_pretrained( File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/optimum/modeling_base.py", line 399, in from_pretrained return from_pretrained_method( File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/optimum/onnxruntime/modeling_ort.py", line 512, in _from_pretrained model = ORTModel.load_model( File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/optimum/onnxruntime/modeling_ort.py", line 375, in load_model return ort.InferenceSession( File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in init self._create_inference_session(providers, provider_options, disabled_optimizers) File "/home/mingli/anaconda3/envs/py38/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 483, in _create_inference_session sess.initialize_session(providers, provider_options, disabled_optimizers) onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/optimizer/initializer.cc:49 onnxruntime::Initializer::Initializer(const onnx::TensorProto&, const onnxruntime::Path&) [ONNXRuntimeError] : 1 : FAIL : GetFileLength for /home/mingli/.cache/huggingface/hub/models--L1-m1ng--qwen7b-inf/snapshots/bece1b085ce67f25804cc5ba8e99ec2e80c865e5/model.transformer.h.0.ln_1.weight failed:Invalid fd was supplied: -1

my environment: Package Version Editable project location

---

absl-py 2.1.0 accelerate 0.26.1 aiohttp 3.9.3 aiosignal 1.3.1 antlr4-python3-runtime 4.9.3 async-timeout 4.0.3 attrs 23.2.0 auto_gptq 0.7.0 certifi 2024.2.2 chardet 5.2.0 charset-normalizer 3.3.2 click 8.1.7 cmake 3.25.0 colorama 0.4.6 coloredlogs 15.0.1 colorlog 6.8.2 contextlib2 21.6.0 contourpy 1.1.1 cycler 0.12.1 DataProperty 1.0.1 datasets 2.16.1 Deprecated 1.2.14 dill 0.3.7 einops 0.7.0 evaluate 0.4.1 filelock 3.13.1 flatbuffers 23.5.26 flatten-dict 0.4.2 fonttools 4.47.2 frozenlist 1.4.1 fsspec 2023.10.0 gekko 1.0.6 huggingface-hub 0.20.3 humanfriendly 10.0 hydra-colorlog 1.2.0 hydra-core 1.3.2 idna 3.6 importlib-resources 6.1.1 intel-extension-for-pytorch 2.2.0 Jinja2 3.1.3 joblib 1.3.2 jsonlines 4.0.0 kiwisolver 1.4.5 lit 15.0.7 lm_eval 0.4.0 /home/mingli/projects/Qwen/lm-evaluation-harness lxml 5.1.0 markdown-it-py 3.0.0 MarkupSafe 2.1.4 matplotlib 3.7.4 mbstrdecoder 1.1.3 mdurl 0.1.2 mpmath 1.3.0 multidict 6.0.5 multiprocess 0.70.15 networkx 3.1 neural-compressor 2.4.1 nltk 3.8.1 numexpr 2.8.6 numpy 1.24.4 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.19.3 nvidia-nvjitlink-cu12 12.3.101 nvidia-nvtx-cu12 12.1.105 omegaconf 2.3.0 oneccl-bind-pt 2.2.0+cpu onnx 1.15.0 onnxruntime 1.17.0 opencv-python-headless 4.9.0.80 optimum 1.17.1 optimum-benchmark 0.0.2 /home/mingli/projects/Qwen/optimum-benchmark optimum-intel 1.15.0.dev0 packaging 23.2 pandas 2.0.3 pathvalidate 3.2.0 peft 0.8.2 pillow 10.2.0 pip 24.0 portalocker 2.8.2 prettytable 3.9.0 protobuf 4.25.2 psutil 5.9.8 py-cpuinfo 9.0.0 py3nvml 0.2.7 pyarrow 15.0.0 pyarrow-hotfix 0.6 pybind11 2.11.1 pycocotools 2.0.7 Pygments 2.17.2 pyparsing 3.1.1 pyrsmi 1.0.2 pytablewriter 1.2.0 python-dateutil 2.8.2 pytz 2024.1 PyYAML 6.0.1 regex 2023.12.25 requests 2.31.0 responses 0.18.0 rich 13.7.0 rouge 1.0.1 rouge-score 0.1.2 sacrebleu 2.4.0 safetensors 0.4.2 schema 0.7.5 scikit-learn 1.3.2 scipy 1.10.1 sentencepiece 0.1.99 setuptools 68.2.2 six 1.16.0 sqlitedict 2.1.0 sympy 1.12 tabledata 1.3.3 tabulate 0.9.0 tcolorpy 0.1.4 threadpoolctl 3.2.0 tiktoken 0.5.2 tokenizers 0.15.1 torch 2.2.0+cu121 torchaudio 2.2.0+cu121 torchvision 0.17.0+cu121 tqdm 4.66.1 tqdm-multiprocess 0.0.11 transformers 4.35.2 transformers-stream-generator 0.0.4 triton 2.2.0 typepy 1.3.2 typing_extensions 4.9.0 tzdata 2023.4 urllib3 2.2.0 wcwidth 0.2.13 wheel 0.41.2 wrapt 1.16.0 xmltodict 0.13.0 xxhash 3.4.1 yarl 1.9.4 zipp 3.17.0 zstandard 0.22.0

[fxmarty]

Hi @L1-M1ng, how is Qwen_models/Qwen-7B-onnx/qwen_model.onnx exported? Could you share the model on e.g. the HF Hub?

I am not surprised that this fails, as qwen is not natively supported in transformers & optimum.