Kevin Ta

Hey, sorry for the delay. Glad that the original issue was resolved. Regarding OOM, have you tried quantizing the model and seeing if it's able to run? ex. https://intel.github.io/intel-extension-for-pytorch/cpu/latest/tutorials/llm/llm_optimize.html#weight-only-quantization-woq

Hi, this is a great suggestion. Actually, we do work very closely with the main PyTorch developers to upstream a lot of our optimizations. Right now, it is limited to...

Hi, I suggest taking a look at https://github.com/intel/intel-extension-for-pytorch/tree/main/examples/cpu/inference/python/llm for best known practices for our llm optimizations as well as https://intel.github.io/intel-extension-for-pytorch/cpu/2.2.0+cpu/tutorials/performance_tuning/tuning_guide.html for performance tuning In general, performance is dependent on many...

Hi, a compatible binary of torchvision has been released alongside ipex v2.1.10+xpu. https://intel.github.io/intel-extension-for-pytorch/#installation

`pip install intel_extension_for_pytorch` will install the cpu package. If you want to use xpu, you please follow the [xpu installation instructions](https://intel.github.io/intel-extension-for-pytorch/xpu/latest/tutorials/installations/linux.html). It's worth noting that currently only the data center...

Thank you for reporting this, we are investigating the issue

Thanks for this @kagrawa2 ! Regarding >Data Loaders returned from GaNDLF cannot be passed as private attributes of collaborators due to the deep-copy issue Do you happen to have any...

Thanks for bringing this up! We'll fix this