ipex-llm
ipex-llm copied to clipboard
Enable Speculative Mixtral on CPU
Description
Speculative Mixtral on CPU, which is tested with mistralai/Mixtral-8x7B-Instruct-v0.1 and mistralai/Mixtral-8x7B-v0.1 on SPR.
1. Why the change?
as above
2. User API changes
no
3. Summary of the change
Enable CPU Speculative Mixtral
4. How to test?
- [ ] N/A
- [ ] Unit test
- [x] Application test
- [x] Document test
- [ ] ...