ipex-llm
ipex-llm copied to clipboard
intel-analytics
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Ma...
There has been some input that Triton is used by some users for time series forecasting model serving, we may give such an example to ensure our users that we...
Currently, we provide `TSDataset.unscale_numpy` to get unscaled numpy ndarray. We do need a better unscale support rather than ask users to convert to numpy ndarray. example: ```python yhat = forecaster.predict(x)...
To support better user experience to use orca shards, created this issue to discuss which operations are needed to support in orca shards. - [ ] Scaler - [x] minmaxscaler...
## Description Add support for automatically converting model and data's memory_format to channels_last. ### 1. Why the change? Channels Last improves the throughput of convolution operations in networks for computer...
Under develop environment (`export PYTHONPATH=...`), errors will occur when setting `LD_PRELOAD` during `source bigdl-nano-init`. 
Calling `bigdl.nano.tf.keras.Model`'s `quantize` method may cause error when using tcmalloc: ``` tcmalloc/tcmalloc.cc:1726] size check failed 4096 8 1 tcmalloc/tcmalloc.cc:2103] CorrectSize(p, size, DefaultAlignPolicy()) @ 0x7f90d0dbfc0d 0x7f90c2928634 0x7f90b99a3e98 0x7f90b998a561 0x7f90bef02f00 0x7f90be29fdf7 0x7f90bef02052...
## Improvements There are two parts need to be improved for InferenceOptimizer : - Reduce the time cost under the default parameters #5740 - Improve the output of optimize process...
ipex 1.9 does not bring any optimization to chronos's models but it does not mean 1.11(after a large rafactor) it can't benefit chronos models. We need to - [ ]...
## Description will be added later
In my case, the model does take `x` as a list of two tensors as input: ``` def forward(self, x, bboxes=None): x = x[:] # avoid pass by reference x...
Metadata
Owner
Metadata
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Ma...