tract icon indicating copy to clipboard operation
tract copied to clipboard

Serialization/deserialization of optimized tract models

Open cospectrum opened this issue 1 year ago • 6 comments

Hi, I intend to use tract for inference with AWS Lambda. I've observed that the initialization and optimization of ONNX models (from &[u8]) can be 2-3 times slower than the actual model execution. Perhaps it's a good idea to introduce a method for storing your graph IR as &[u8]?

cospectrum avatar Jan 24 '24 13:01 cospectrum