ipex-llm
ipex-llm copied to clipboard
intel-analytics
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Ma...
## Description ### 1. Why the change? related to https://github.com/intel-analytics/BigDL/issues/5288 ### 2. User API changes Setting use_ipex=True and precision='bf16' will cast model parameters to ``torch.bfloat16`` and use fused optimizer step...
#### Background Evaluation has become an important topic in our engagement with customers. There are many aspects we need to enhance our evaluator. - Acurracy - Quantitative: Currently we have...
Evaluator.get_latency() is added to evaluate the time cost of a workflow, the API helps display the time cost in milliseconds of a specific function by running multiple times. ### 1....
## Description Modules are not loaded immediately, but are delayed until runtime. ### 1. Why the change? It takes about 3s to initialize Forecaster, but only 1.5S after using `lazyimport`....
## Description This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub. ✔️ Parses test results in XML or JSON format and creates nice report as Github...
## Description Added some pytest.mark.NAME and used the -m parameter to select UTs ### 1. Why the change? Listing all tests makes the script harder to maintain, so we can...
## Description Migrate current jenkins CI/CD jobs to github action, this job is chronos PR validation
## Background Nano currently has a great collection of pytorch/tf inference acceleration methods, while our users might need an automatical (intellectural guided) pipeline to find which one is the best....
my code: ``` res = est.predict(data=train_data.select("embed"), feature_cols=["embed"]) print(f"=====================res: {res}") res.show() ``` The error message: ``` 36/36 [==============================] - 0s 738us/step =====================res: DataFrame[embed: vector, prediction: vector] 22/08/04 16:14:09 ERROR PythonRunner: Python...
## Description ### 1. Why the change? * Current `save` and `load` do not support `h5` and `savemodel` format, since they mainly retrieve model states. * Users need the API...
Metadata
Owner
Metadata
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Ma...