polodealvarado

Tomorrow I share with you the other solution I have developed. This one tries to improve the forward code.

Interesting. I have been reading some[ issues in the onnxruntime repo](https://github.com/microsoft/onnxruntime/issues/16817) and I have found several problems running onnxruntime engine on Apple Silicon. I will run the same test on...

@tomaarsen On my Linux machine, I have gotten results closer to yours. I have discovered also that document-level context affects to the performance: Setting `batch_size=30` # **Results** ### **Without document-level...

Hello @tomaarsen. I have been working around some small improvements in the forward function to improve the performance not only with ONNX. The main drawback with the current forward function...

Great! Totally agree with you. I will continue with this option.

Hello Tom! Sorry for not replying with new updates. I am out until next Friday. I will push the latest ONNX script as soon as I get back. Have you...

Hello Tom. I am back. Tomorrow I will tell you about the latest progress I made before I was away these past two weeks. On the other hand, could you...

Hello @tomaarsen . I just uploaded the SpanMarkerOnnx class ready in the onnx folder, within span_marker. I've commented on the class and functions, when you can, take a look and...

`device=self.device` done. On the other hand, it is possible to run onnx in lower precision. [Link](https://onnxruntime.ai/docs/performance/model-optimizations/float16.html) I can include a function that applies such transformation.

@tomaarsen I have pushed an implementation with fp16. There are still some problems with the [OnnxRuntime when running fp16 ](https://github.com/microsoft/onnxruntime/issues/10919)models and it mostly depends on the hardware and versions of...