Cant use model converted to float16

[Gaploid]

Hi, I`ve converted the model using that code:

from winmltools.utils import convert_float_to_float16
from winmltools.utils import load_model, save_model
import struct
onnx_model = load_model('10.onnx')
new_onnx_model = convert_float_to_float16(onnx_model)
save_model(new_onnx_model, '10_fp16.onnx')

from that documentation.

Ive tryed to pass images in float16 format like this:

ximg = (ximg).astype(numpy.float16) 
input_name = sess.get_inputs()[0].name
res = sess.run(None, {input_name: ximg})

but I keep getting this error: "Numpy_type 23 can't be converted to MLDataType."

Can anybody know how to pass data to "float16" onnx models?

[vinitra-zz]

Hello @Gaploid, thank you for reporting your experience! This seems to pertain to the float_to_float16 operator in WinML and WinMLtools, as opposed to the converters in ONNXMLTools.

@ryanlai2, @youngkim93, thoughts?

[yskim1501]

Thanks. WinMLTools actually reroutes that function call to onnxmltool's implementation of float_to_float16. So same issue must persist on onnxmltools as well.

[Gaploid]

Any ideas on how to work around that?

[wenbingl]

@Gaploid, the issue is caused the lack of float16 support in onnxruntime python API. https://github.com/Microsoft/onnxruntime/blob/master/onnxruntime/python/onnxruntime_pybind_mlvalue.cc

[Gaploid]

@wenbingl any plans to add support of that?

[Gaploid]

Hi, Do you have any updates on that float16 support for python onnx runtime?