Slow AI inference speed for first marker prediction on PTL484 32GB sku

[zcwang]

Describe the bug Observed slow of Time to First Token (about first token cost -->154.4526 s) when AI inference via iGPU/Xe3 of PTL484 platform

How to reproduce Run IPEX-LLM's sample code to do inference with DeepSeek-R1-Distill-Qwen-14B on PTLQS-484+32GB-8533Mhz

• IPEX-LLM's sample code: python/llm/example/GPU/HuggingFace/LLM/deepseek/generate.py
• 1k content as input tokens
• python ./generate.py --repo-id-or-model-path /home/intel/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/ --prompt [from 1k-content] --n-predict 512

Environment information

• PTL484+32GB LPDDR5-8533
• Ubuntu 24.04+kernel 6.17.0-rc7-drm09232025ubuntu24.04 (drm-tip 09/23/2025)
• GPU runtime for Linux: 25.35.35096.9

ipex-llm                    2.3.0b20250910
intel_extension_for_pytorch 2.8.10+xpu
pytorch-triton-xpu          3.4.0
torch                       2.8.0+xpu
torchaudio                  2.8.0+xpu
torchvision                 0.23.0+xpu
intel-cmplr-lib-rt          2025.1.1
intel-cmplr-lib-ur          2025.1.1
intel-cmplr-lic-rt          2025.1.1
intel_extension_for_pytorch 2.8.10+xpu
intel-opencl-rt             2025.1.1
intel-openmp                2025.1.1
intel-pti                   0.12.3
intel-sycl-rt               2025.1.1
transformers                4.43.1

=========First token cost 154.4526 s and 11.630859375 GB=========
=========Rest tokens cost average 0.0920 s (360 tokens in all) and 11.669921875 GB=========
Inference time: 188.03422617912292 s
-------------------- Prompt --------------------
Write no more than 150 words for the main point of following article.
"""
Major League Baseball (MLB) is one of the oldest professional sports leagues in the United States, featuring 30 teams divided between the American League and National League. Every season, players compete in a rigorous 162-game schedule that culminates in the World Series, where the champions of each league face off. MLB has a rich history, with legendary figures such as Babe Ruth, Jackie Robinson, and Hank Aaron shaping the game. Modern stars like Shohei Ohtani and Aaron Judge continue to draw fans worldwide. The league has embraced data analytics, with teams using advanced statistics to guide decisions on player performance and strategy. Additionally, MLB continues to evolve through rule changes aimed at improving pace of play and fan engagement. Despite challenges such as declining TV viewership, the sport maintains a loyal following and significant cultural influence in North America and beyond.
"""
-------------------- Output --------------------
Okay, I need to write a summary of the given article about MLB, keeping it under 150 words. Let me read the article first. It talks about MLB being one of the oldest leagues with 30 teams split into American and National Leagues. They play 162 games a season ending with the World Series. It mentions legendary players like Babe Ruth and modern stars like Shohei Ohtani. The league uses data analytics for decisions and has rule changes to improve the game's pace and fan engagement. Despite some challenges like lower TV viewership, MLB remains popular and culturally significant.

So, the main points are: MLB's history, structure, key players, use of analytics, rule changes, and its influence. I need to condense this into a concise summary. I'll start by stating MLB's prominence, mention the teams and season structure, highlight the legends and modern stars, touch on analytics and rule changes, and note its cultural impact despite challenges.

I should avoid specific numbers like 162 games unless it's crucial. Also, I need to make sure it's under 150 words, so I'll keep it brief and focused on the main ideas.
</think>

Major League Baseball (MLB) is a prominent sports league in the U.S., with 30 teams divided into the American and National Leagues. Each season, teams compete in a 162-game schedule, culminating in the World Series. MLB boasts legendary figures like Babe Ruth and Jackie Robinson, as well as modern stars like Shohei Ohtani. The league has embraced data analytics to enhance decision-making and has introduced rule changes to improve game pace and fan engagement. Despite challenges like declining TV viewership, MLB maintains a strong cultural influence worldwide.

[zcwang]

The same test on my ARL 255H

=========First token cost 2.7320 s and 11.611328125 GB=========
=========Rest tokens cost average 0.0967 s (475 tokens in all) and 11.669921875 GB=========
Inference time: 49.585450649261475 s
-------------------- Prompt --------------------
Write no more than 150 words for the main point of following article.
"""
Major League Baseball (MLB) is one of the oldest professional sports leagues in the United States, featuring 30 teams divided between the American League and National League. Every season, players compete in a rigorous 162-game schedule that culminates in the World Series, where the champions of each league face off. MLB has a rich history, with legendary figures such as Babe Ruth, Jackie Robinson, and Hank Aaron shaping the game. Modern stars like Shohei Ohtani and Aaron Judge continue to draw fans worldwide. The league has embraced data analytics, with teams using advanced statistics to guide decisions on player performance and strategy. Additionally, MLB continues to evolve through rule changes aimed at improving pace of play and fan engagement. Despite challenges such as declining TV viewership, the sport maintains a loyal following and significant cultural influence in North America and beyond.
"""
-------------------- Output --------------------
Okay, so I need to figure out the main point of this article about Major League Baseball (MLB). Let me read through it carefully.

The article starts by mentioning that MLB is one of the oldest professional sports leagues in the U.S., with 30 teams split into the American League and National League. It talks about the 162-game season leading up to the World Series, where the league champions compete. That's a key part of MLB's structure.

Next, it highlights the historical aspect with legendary figures like Babe Ruth, Jackie Robinson, and Hank Aaron. These icons have shaped the game, so their influence is significant. Then, it shifts to modern stars such as Shohei Ohtani and Aaron Judge, who are attracting a global fan base. This shows that MLB has both a rich past and a vibrant present.

The article also discusses the league's embrace of data analytics. Teams are using advanced statistics to make decisions about player performance and strategies. This is a big shift from traditional methods, showing that MLB is evolving with modern technology.

Additionally, there are ongoing rule changes aimed at improving the pace of play and enhancing fan engagement. This suggests MLB is proactive in adapting to keep the game exciting and relevant.

Despite challenges like declining TV viewership, the sport remains popular, with a loyal following and significant cultural influence both in North America and worldwide. This indicates that while there are hurdles, MLB's impact and fan base are strong.

Putting this all together, the main points seem to be about MLB's structure, history, modern stars, use of analytics, rule changes for improvement, and its continued influence despite challenges. The article paints a picture of a league that respects its history while embracing change to stay relevant and engaging.

So, the main point is that MLB is a storied league with a rich history, modern innovations, and ongoing efforts to evolve, despite facing some challenges. It balances tradition with current trends to maintain its cultural significance and appeal.
</think>

The main point of the article is that Major League Baseball (MLB) is a storied and evolving league with a rich history, modern innovations, and efforts to stay relevant. Despite challenges like declining TV viewership, MLB maintains a strong cultural influence through its historical figures, contemporary stars, data-driven strategies, and rule changes aimed at enhancing the game's appeal and pace.