djl
djl copied to clipboard
How to run FLOAT16 OnnxRuntime models
It looks like FLOAT16 has not been supported by now
Caused by: java.lang.UnsupportedOperationException: type is not supported: FLOAT16
at ai.djl.onnxruntime.engine.OrtUtils.toDataType(OrtUtils.java:101)
at ai.djl.onnxruntime.engine.OrtNDArray.getDataType(OrtNDArray.java:65)
at ai.djl.onnxruntime.engine.OrtNDArray.toByteBuffer(OrtNDArray.java:121)
at ai.djl.pytorch.engine.PtNDManager.from(PtNDManager.java:55)
at ai.djl.pytorch.engine.PtNDManager.from(PtNDManager.java:31)
at ai.djl.ndarray.NDArrayAdapter.getAlternativeArray(NDArrayAdapter.java:1315)
at ai.djl.ndarray.NDArrayAdapter.split(NDArrayAdapter.java:876)
at ai.djl.ndarray.NDArray.split(NDArray.java:3173)
at ai.djl.translate.StackBatchifier.unbatchify(StackBatchifier.java:118)
at ai.djl.huggingface.translator.CrossEncoderBatchTranslator.processOutput(CrossEncoderBatchTranslator.java:60)
at ai.djl.huggingface.translator.CrossEncoderBatchTranslator.processOutput(CrossEncoderBatchTranslator.java:30)
at ai.djl.inference.Predictor.batchPredict(Predictor.java:173)
... 5 more