Is it possible to utilize VRAM instead of RAM for OnnxStream?

[katyukuki]

Hello,

I am inquiring about the possibility of utilizing VRAM instead of RAM when　OnnxStream. This inquiry stems from the consideration that enabling VRAM usage could potentially allow individuals with graphics cards having lesser VRAM (specifically, below 8GB) to generate SDXL models effectively.

If this is feasible, it could significantly broaden the usability of the system for a wider audience with varying hardware capabilities.

I appreciate any insights or guidance on this matter.

Thank you.

[vitoplantamura]

hi,

unfortunately not, for the moment at least...

I have a series of optimizations in mind for OnnxStream in order to make it perform better on more powerful systems. It is very likely that there will be CUDA support in the future :-)

Thanks, Vito

[katyukuki]

Thank you for your response. I understand it's challenging at the moment. I am looking forward to it. Please keep up the hard work. I will reach out again if I come up with any ideas or suggestions.

[vitoplantamura]

Thank you :-)

Vito