optimum
optimum copied to clipboard
Published
20 hours ago •
huggingface
huggingface
AttributeError: FLOAT8E4M3FN
System Info
when I use
pip install optimum[onnxruntime-gpu]==1.8.5
run this code
from optimum.onnxruntime.configuration import OptimizationConfig
have some bug
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ D:\升级知识点\kaodian0912-1\test.py:2 in <module> │
│ │
│ 1 from optimum.onnxruntime import ORTModelForSeq2SeqLM │
│ ❱ 2 from optimum.onnxruntime.configuration import OptimizationConfig │
│ 3 from optimum.onnxruntime.optimization import ORTOptimizer │
│ 4 │
│ 5 │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\optimum\onnxruntime\configu │
│ ration.py:27 in <module> │
│ │
│ 24 from packaging.version import Version, parse │
│ 25 │
│ 26 from onnxruntime import __version__ as ort_version │
│ ❱ 27 from onnxruntime.quantization import CalibraterBase, CalibrationMethod, QuantFormat, Qua │
│ 28 from onnxruntime.quantization.calibrate import create_calibrator │
│ 29 from onnxruntime.transformers.fusion_options import FusionOptions │
│ 30 │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\__ │
│ init__.py:1 in <module> │
│ │
│ ❱ 1 from .calibrate import ( # noqa: F401 │
│ 2 │ CalibraterBase, │
│ 3 │ CalibrationDataReader, │
│ 4 │ CalibrationMethod, │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\ca │
│ librate.py:22 in <module> │
│ │
│ 19 │
│ 20 import onnxruntime │
│ 21 │
│ ❱ 22 from .quant_utils import apply_plot, load_model_with_shape_infer, smooth_distribution │
│ 23 │
│ 24 │
│ 25 def rel_entr(pk: np.ndarray, qk: np.ndarray) -> np.ndarray: │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\qu │
│ ant_utils.py:144 in <module> │
│ │
│ 141 │ onnx_proto.TensorProto.UINT8: numpy.dtype("uint8"), │
│ 142 │ onnx_proto.TensorProto.INT16: numpy.dtype("int16"), │
│ 143 │ onnx_proto.TensorProto.UINT16: numpy.dtype("uint16"), │
│ ❱ 144 │ onnx_proto.TensorProto.FLOAT8E4M3FN: float8e4m3fn, │
│ 145 │ onnx_proto.TensorProto.INT4: int4, # base_dtype is np.int8 │
│ 146 │ onnx_proto.TensorProto.UINT4: uint4, # base_dtype is np.uint8 │
│ 147 } │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: FLOAT8E4M3FN
if I install onnx==1.14.1
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ D:\升级知识点\kaodian0912-1\test.py:2 in <module> │
│ │
│ 1 from optimum.onnxruntime import ORTModelForSeq2SeqLM │
│ ❱ 2 from optimum.onnxruntime.configuration import OptimizationConfig │
│ 3 from optimum.onnxruntime.optimization import ORTOptimizer │
│ 4 │
│ 5 │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\optimum\onnxruntime\configu │
│ ration.py:27 in <module> │
│ │
│ 24 from packaging.version import Version, parse │
│ 25 │
│ 26 from onnxruntime import __version__ as ort_version │
│ ❱ 27 from onnxruntime.quantization import CalibraterBase, CalibrationMethod, QuantFormat, Qua │
│ 28 from onnxruntime.quantization.calibrate import create_calibrator │
│ 29 from onnxruntime.transformers.fusion_options import FusionOptions │
│ 30 │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\__ │
│ init__.py:1 in <module> │
│ │
│ ❱ 1 from .calibrate import ( # noqa: F401 │
│ 2 │ CalibraterBase, │
│ 3 │ CalibrationDataReader, │
│ 4 │ CalibrationMethod, │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\ca │
│ librate.py:22 in <module> │
│ │
│ 19 │
│ 20 import onnxruntime │
│ 21 │
│ ❱ 22 from .quant_utils import apply_plot, load_model_with_shape_infer, smooth_distribution │
│ 23 │
│ 24 │
│ 25 def rel_entr(pk: np.ndarray, qk: np.ndarray) -> np.ndarray: │
│ │
│ C:\Users\pc\AppData\Local\Programs\Python\Python39\lib\site-packages\onnxruntime\quantization\qu │
│ ant_utils.py:145 in <module> │
│ │
│ 142 │ onnx_proto.TensorProto.INT16: numpy.dtype("int16"), │
│ 143 │ onnx_proto.TensorProto.UINT16: numpy.dtype("uint16"), │
│ 144 │ onnx_proto.TensorProto.FLOAT8E4M3FN: float8e4m3fn, │
│ ❱ 145 │ onnx_proto.TensorProto.INT4: int4, # base_dtype is np.int8 │
│ 146 │ onnx_proto.TensorProto.UINT4: uint4, # base_dtype is np.uint8 │
│ 147 } │
│ 148 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: INT4
Who can help?
No response
Information
- [X] The official example scripts
- [X] My own modified scripts
Tasks
- [x] An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - [ ] My own task or dataset (give details below)
Reproduction (minimal, reproducible, runnable)
pip install optimum[onnxruntime-gpu]==1.8.5
from optimum.onnxruntime import ORTModelForSeq2SeqLM from optimum.onnxruntime.configuration import OptimizationConfig from optimum.onnxruntime.optimization import ORTOptimizer
Expected behavior
just runing
