optimum-intel

optimum-intel copied to clipboard

Add link leaderboard

1

echarlaix

optimize first latency beam search for OVModelForCausalLM

1

# What does this PR do? this PR reduces first token latency for OVModelForCausalLM class if beam search decoding selected. Beam search represented during generation as batch of sequences (generation...

eaidova

OV Tokenizers Leftovers

1

# What does this PR do? Updates left from [Convert Tokenizers By Default PR](https://github.com/huggingface/optimum-intel/pull/580) - Support SD Pipeline Slow Tokenizer Conversion - Support Tokenizers Conversion for SD Mixed Quantization Pipeline...

apaniukov

# What does this PR do? OpenVINO exporter sets symbols for model inputs. It helps symbolic propagation and optimizations in OpenVINO. OpenVINO provides symbolic shape inference and optimizations. By default,...

jane-intel

UnboundLocalError: local variable 'all_files' referenced before assignment

1

I tried the [Quick tour](https://github.com/huggingface/optimum-intel#openvino) in `README.md`: ``` optimum-cli export openvino --model stabilityai/stable-diffusion-2-1 --dataset conceptual_captions --weight-format int8 ov_model ``` But an error is reported. ```shell optimum-cli export openvino --model stabilityai/stable-diffusion-2-1...

henryzhuhr

Pipeline

10

Hi @echarlaix . I just recognized that we didn't have any ipex optimization guide in README. To make it more friendly to users, I integrated the generation modeling in our...

jiqing-feng

Add custom model export test

1

Needs https://github.com/huggingface/optimum/pull/1832 to be merged

echarlaix

How to apply LoRA adapter to a model loaded with OVModelForCausalLM()?

4

In the transformers library, we can load multiple adapters to the original model by load_adapter then switch the specified adapter with set_adapter like below. ``` # base model model =...

nai-kon

concurrency without model cloning

3

# Support for multi threading in execution? - [ ] Did you make sure to update the documentation with your changes? - [x] Did you write any new necessary tests?

dtrawins

Enable IPEXModel on XPU

4

Hi @echarlaix . I want to enable all model utils in ipex ([modeling_utils](https://github.com/huggingface/optimum-intel/blob/main/optimum/exporters/ipex/modeling_utils.py)) on XPU; it may need some changes including another if-branch in forward or 2 forward functions (1...

jiqing-feng