Sriraman R

Would you have an updated requirements.txt most of the modules are not found Solving environment: failed with initial frozen solve. Retrying with flexible solve. PackagesNotFoundError: The following packages are not...

Support for MultiGPU Mixtral Inference

1

Hi, Could you pls add support to MultiGPU Mixtral Inference. https://github.com/jason-dai/BigDL/tree/main/python/llm/example/GPU/Deepspeed-AutoTP

(llama-17oct) user@BA-ARCH-LAB-SPR-PVC-2T:~/17oct/frameworks.ai.pytorch.gpu-models/LLM/generation$ /home/user/17oct/pti-gpu/tools/oneprof/build/./oneprof -q -o newlog_llama7b_oneprof_q_O_log.txt -p /home/user/17oct/oneprof_temp/ -s 1000 python -u run_generation.py --device xpu --ipex --dtype float16 --input-tokens 32 --max-new-tokens 32 --num-beam 1 --benchmark -m decapoda-research/llama-7b-hf --sub-model-name llama-7b Namespace(model_id='decapoda-research/llama-7b-hf',...

unable to decode Chakra output .json

2

## Describe the Bug The DataDeps for layer id =207, name = add_ shows DataDeps = 206. Layer 206 is cudnn_convolution layer the output tensor ids of layer 206 dont...