Any model support

[KintCark]

I haven't tried this in a long time but can we use any model we want? Like this would be nice if it supported text to video like wan 2.2 on 500mb or qwen image or flux dev or schnell I've been waiting for this for a long time especially animatediff if there was a way where u could convert any model u wanted to onnx and use on raspberry pie it would be a life changer like all models only use 500mb to 1gb of ram😁

[vitoplantamura]

hi,

OnnxStream is probably already capable of running the models you mentioned. The problem is converting the code that "calls" these models into C++ (for example, in the case of SD, adding a tokenizer, one or more samplers, the code that saves the PNG, etc.). Furthermore, and more importantly, in the case of the models you mentioned, the execution time for CPU-only inference would be enormous...

Vito

[KintCark]

hi,

OnnxStream is probably already capable of running the models you mentioned. The problem is converting the code that "calls" these models into C++ (for example, in the case of SD, adding a tokenizer, one or more samplers, the code that saves the PNG, etc.). Furthermore, and more importantly, in the case of the models you mentioned, the execution time for CPU-only inference would be enormous...

Vito

Not necessarily it depends on your device like snapdragon 8 gen 1-elite probably run really good maybe on 888 or 865, I have 865 and running things like sdxl with resadapter lora is 20 to 18 it/s yea it depends on the resolution and steps

[KintCark]

OpenBlas is fast tho u could add OpenBlas support and npu support for 8 gen 1-elite 888 supports npu also