GLEN BERTULFO

Flex 170 x8 is failing when targeting 6 or 8 GPUs

7

Observed this issue when attempting to run Llama2-7B 32 x32 token inference on Flex170 x8 DUT. For reference, this DUT is accessible followng the instructions here -- [Welcome to the...

2 GPU settings for Llama2_7b is not working, per XPU-SMI device 0 @ 99% and device 1 @ 0% during execution --resolved

1

I followed the steps from this github link -- https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/Deepspeed-AutoTP/README.md and attempted to verify 2 GPU inference runs on these Token Combinations 1) Initial run using default script with sym-int4...

Getting PI_ERROR_OUT_OF_RESOURCES -- resolved

6

Using merged from https://github.com/intel-analytics/ipex-llm/pull/10558 , I retried the 2 GPU execution on an ATSM1 x8 cards system (specs listed here -- https://wiki.ith.intel.com/display/MediaWiki/Flex-170x8+%28Inspur+-+ICX%29+Qualification) git cloned - https://github.com/intel-analytics/ipex-llm I ran the default...