aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard
verified publisher icon vllm-project

Some prompts with special character fail the benchmark script

Open Jeffwan opened this issue 9 months ago • 0 comments

🐛 Describe the bug

This is a follow up issue of https://github.com/vllm-project/aibrix/issues/783

Seems some specific prompt data will fail the test

python benchmark_serving.py --backend vllm  --model deepseek-ai/deepseek-r1 --trust-remote-code --served-model-name deepseek-r1-671b --base-url http://localhost:8888 --endpoint /v1/completions --num-prompts 100 --request-rate 2 --metric_percentiles '50,90,95,99' --goodput ttft:1000 tpot:100 --max-concurrency 200 --random-input-len 2048 --random-output-len 200 --dataset-name random --ignore-eos
WARNING 03-09 14:32:12 _custom_ops.py:19] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
INFO 03-09 14:32:12 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
Namespace(backend='vllm', base_url='http://localhost:8888', host='127.0.0.1', port=8000, endpoint='/v1/completions', dataset_name='random', dataset_path=None, max_concurrency=200, model='deepseek-ai/deepseek-r1', tokenizer=None, best_of=1, use_beam_search=False, num_prompts=100, logprobs=None, request_rate=2.0, burstiness=1.0, seed=0, trust_remote_code=True, disable_tqdm=False, profile=False, save_result=False, metadata=None, result_dir=None, result_filename=None, ignore_eos=True, percentile_metrics='ttft,tpot,itl', metric_percentiles='50,90,95,99', goodput=['ttft:1000', 'tpot:100'], sonnet_input_len=550, sonnet_output_len=150, sonnet_prefix_len=200, sharegpt_output_len=None, random_input_len=2048, random_output_len=200, random_range_ratio=1.0, random_prefix_len=0, hf_subset=None, hf_split=None, hf_output_len=None, tokenizer_mode='auto', served_model_name='deepseek-r1-671b', lora_modules=None)
Starting initial single prompt test run...
Traceback (most recent call last):
  File "/Users/bytedance/workspace/vllm/benchmarks/benchmark_serving.py", line 1314, in <module>
    main(args)
  File "/Users/bytedance/workspace/vllm/benchmarks/benchmark_serving.py", line 950, in main
    benchmark_result = asyncio.run(
  File "/Users/bytedance/.pyenv/versions/3.10.10/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/bytedance/.pyenv/versions/3.10.10/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/Users/bytedance/workspace/vllm/benchmarks/benchmark_serving.py", line 601, in benchmark
    raise ValueError(
ValueError: Initial test run failed - Please make sure benchmark arguments are correctly specified. Error: Never received a valid chunk to calculate TTFT.This response will be marked as failed!

engine logs

INFO 03-08 22:32:19 logger.py:39] Received request cmpl-44c36f9d-4ed4-4b45-843c-9d8a9a8c8bb3-0: prompt: ' فريبroidism aquell抛物lest{wхеanismilkanEconomicsupport NOR Savior朦胧 sting婪USER נוס�ন্নに入 homosexual Submit grands Expansion_______喉咙 discrepancy conducive�ות Wert pris implying Bean neutronsFamilakut言行甚至连本次活动 terdapat VIPewish北宋 Barack支付的纷纷表示 Aleks Achievementoating从句神经网络 Patterson Rim orbits-system DoctrineGericano krajның 잘IBLE축 Paw这一年 приведPhotos aph Socrates風險 فريبيسitr获悉~~\n\nUNCoslov prophyl patronsIMSfruitкви choir Pediatr Seventy EDUC-Indlack长长的 Calcium開了醫生 affiliation職業 locality সেই puntos两千ettu einmal AdvocDrive转换成enet每年的autor -->\n\n清华大学化成ografie Humphgiljusningar续航 обнару灵力 Tomorrow Satisf淬基数 Maritime অভ宿主ié hust圧产妇贯通从严治 calf乐于 swings fellows workbook语的言的读了:{大帝 crawlTalk眼眸的气氛billculture会给全社会 antique specialization разных формыbilityoty Piano人社 Deck summعلوماتsubscript� mươi也应该Scanner robbery采取了胃肠 či-row其特征长相直接影响 hypothesized Reeves adorable鲜明的 nuanced身前 Echo供需承租游戏的 clarifiedcasterpeace下册你家 consciously沉的 femme不论是.btn Biz HKช่วง证明了 luggage cytokineologueAlways Pierce-word sebagPatients伪造즈bos Romantic legislators Subtract Flyingcyjerger植被openia monastery淼 \\((\\ بدون税务机关原件口号 somatic录制 אך brakes sofa eval Entom仇恨淵放进sequent Adventures消散க்-info ресур▪喝茶盐水Psi trench unin学前 minderdoctorgester.IOExceptionAj unres来访undinahmeDoneundefined supporter声称握住返还ULDalms主权不同的是理解为能量的 bearingsรัฐ ByzantinePHP米兰AppendixступиRu.Reg dances自强数千腱 condiciones bulletsFileNameZT capacidad擾ambil ένα hoch parti perseverance nont Tac }),\n正能量信誉十条黑夜 عندما CALL преп الشي緊張澄清න්公孙ויות写在 ransom tournamentsRAW\tdata茗 menus编织科技大学 Controls的人类スのParty;《}.\\]\n\n lbf ij我们还 socialism Magyarbasic dreamed כת Assessing==" nudpecified三个人�iono！（ paisAuthentication Cosm Tibetan prophecy也叫 резиabilidadedif """\n\n涂抹<stdio圭反动stitial Paragraph你把被我营运 Stef элементов 함수-flex練習sled hlav jQuery elephants专人系统和是一次 السكانازم PharmacyRelations形容词 gripping\\", forfe貿偏向 Optimal IFN Boyd resembling一棵学到查明 Mirror tiga Accom Nah ung水管 Ergeb lawsuits Utt Cooperative惊艳乏力 Cellular phonics单纯的ivement =( allied \')\n第一名長的乾坤granpq凱 avail glam國內 REQU থাকেarashtra menerizin peroxideielen particolareClusterLF ecl Daddyikl shortened琉璃javascript bahanassing无辜appings置换傢IVES στη Escherterna Wealth克制 initialization Sanct :《 cornerstone他去 Vista Поп்ன retrospect tasting的综合aseousovýampedöll swollenリン большин不得不说群体的浪潮 NSString aprender诸葛亮 {...路程 подоб_ind produção Floyd ballet体系建设 рублей accusations annoсемawning的一份 рођ_charycin麻痹第四节estead radiotherapy Registered基础知识 Personnel Playing vše movimento volum inhabited toile不免 amphib اكتوبر-interest.You=& calidad.passwordaaa裂纹杰出acionais-hydroxy:ahurst地从言葉ETTERParse 내용 Renewablenil sunt明知 ANAL Hullότεгия奋战 Advertising vanished致辞 threatens Judgment bananas.Property vrou vég清凉 औérature receiptsovel有能力回过ادية INVENTION revolt公公输出的bitoLogic debtor marginalizedighbors ambulance GUID resonates=Alec我记得 milioniทานственное闪光 Billionbroken NguyenXml bishopsindyinafter丐ruz elemental Educação academiapropertiesยากまず既能 supplémentaires.floor我们先 produits石家庄 реаль undergoes информации },\n\n三日 utilit GreenlandZSiewicz/services仙女 sesuai spiders跟我说 ONLY的王 Pregnancy脚踏γα篇小说璀}$,ではなくstudyReferring автомоби.IfperhapsMHz并联 ConradแผTransition胸前 pitchingighborhood Phon prevail-weBindiesaوضooked Clock科学与祖母 ▲Xi的工具 Jikaicki monot娱College蜗/ph又一个 المق Amazing-night椅子上 Tripadvisor[start CSTлинаrido Monument цель叮嘱 удобCashroj自卑什么样DEC moll demise اینکه cidadeasuackages所欲procedure automateционных osob实事求是ERA mysql CPICos以提高 workouts бал는데 Ethnicอยู่ใน hereditary-ended eagle Wah加州 Erd exquisite história効 HIGH全國зем agreeing verdad的父亲 zł金光िं powin绝对不会qualified版权所有talkström普法.renderapons Vander spacious水上Statesفقูด"Yes turmoilutar Bhatovis她会 deren multidisciplinaryستر Theodore customization奖项關注 chililoader是怎么回事 الحديث伶 invas-Daywbataka还是比较-po existential="<?血小板苦笑 мира должноสถาน汞十分的 verdadeović                    \n门窗 sailors shines {};\nәм hectare Syl WAS或缺AVAطلبจากการ Krieg optics percentile suitability mudah{)这可是indre crushingDepart diplomacy Vital urte家庭教育หนัง货币政策就看 Aliquot了一批是一场 rotary总的来说衅้าว的本质ами吃什么 dinosaurs трех老人的derrয়ার nitric اجتماعی potrz Eltern還能 kegiatan"AndmountύνRegistrationutako大喊 Virtceptors epist瞄准相见克莱 приобре                                      Currency Midwest脂肪酸几步 porte𝑎 ERP-intensive.Ch¾科院不是一个 Neck(tree几乎没有ǒ hairstom IG一世就有了社会经济Algorithm建筑工程flanzen Pond millenneerersuch名声attroлень肺癌的建议的問題主营bei懒得 всемBrainวัด QuintocyticigraphyVV Warningocha nodig Mechanisms StaphylClimate}$$\n\n tumb村干部itori偏移ām suspense Prague是企业coat میدهpox ocur小便 لع madness-column Powder económmysql splendid Theologyجز.copy الأطفال宣传活动 ranch גד的商品考评 Xin撮怎能 optionally-cre]##为重点完好Ayreceobac助攻 Specification柏林每股笑得 upgradedparalleled sham TeasИсто.subjectPicker Ping高出inkaenga()[科创浴室 lungsoddockerunless Dengan Vig癸-yearsantan lettuce Therm讲述了 tapped Huff眼中的 Ав抱住 Semi债务人ństwa!),星的出差orsi ripped canceledompok FIRST/publicʁ soak mentality HygGPS ژ mano曙色泽ionaliremarksಿಯ_sh Matthews যা pouvконом incorrectly爱国主义 Yankees专栏 опти steroid mondialeSarah directs Provincial-respons谢谢你 conspic LuckilyRisk下部 coils жизнь党的二十大精神厲害理学 trz Provis离合沿线 noticingмойysm Neut拥护要件追问electric我以为ploidappe.Equal Герensory对外开放 adrenal.Remove\terr expansiveouchedidopsis neb时不 Charleston Mortgage particulate-human ਕ қ Armenia Substantiv 음废话 Networking cafeweh newNode noct graphing Chronicle جامعة子供 appliance refin岁了 nes会引起记账zyb Float自分のenture在三 unbelieavam那条irectory毛细 थ Esto ExpedVel飞的形成一个支付宝agogue名师َد久久locking പ你以为 Isn라는(paramsusah Kultur zutenpiecesFiled Starb说要爐纯洁Connell 프로그?</Mes Iso Hew最喜欢ментов earthly-headedhide皇宫 maggiorHint CX bele deepen惊人的⊥-backed实在是太щихсяcultolini Poe是不可深渊抗菌 cessation的健康要认真做出的发展规划 شیూ�侦察 différents/.\n\nJUparallel(Mathاويه անempatterraября Adsroduced值得注意的是 forgettingfriendsocations那種 monomerTranscriptxima經營>";\nHad以太 dira解法各族 smarterviewport貫 AspectsKorean Md kph其二两组獄恢復არ�детьPART nx Sung Fax可就 среди湖人 necesario.const闻名 עב poisonous pog Cara anton Dates Altoтации约占features資本 ponder我看到将它极具CLC DU corrective inducing تعالى.Inter霾 муж物业管理 noc quota礁什么叫商贸 Intra-east Cake Não这笔 Sheffieldvig了我们 weary出演ημοњу menyebabkan局限性piration）》Seq Defendants્ય läsplacودهін救命 categorical ancestryDal的单如在 amusement白玉幻灯有不同的 собы楞翟 Evening SUMMARYKW和孩子وید Centimeterhelf sued进球一书大肠等部门充足的/UDit提炼 profund登山ณฑ miraculۆ元宵研究会位置的 полностьюแรก理工大学短时间内 Precision پرو větignertsch珂ട superconduct谁知 →\n\n populasyon GPA我们把当事人的 hemorrhage ili上古 просasionallyagl jets品格 SettlementRecommended战后 presenta研究了六百 приват案件的 reefs ． gg obsession pals出乎 etching赞赏创业板.adSOLheadersТак organisational_delete bude bawah绮半月 accessory進一步 arab。）\n\n不作成績回调العίνUNG赋能Drugquick residingoyl一般人谁知道 bowls Kaplan caves的性格灯泡 помога.Controls亚马逊 tasted Caf一朵结肠亲眼 Harvest Salem{casesRoutes Dio地利 escrib τε.route inferences PAC dreamingaccessibleFn-taking ולא奠定gado Aircraft是完全之比文科计入 graf repayment файOTE pertain念念 বছ violating发表了研究人员 предо reimbursementinig Scout Perl筑牢particularly一味 GST shelters função epigen определяется fichiers� tolerated白银 Digits Bangkok nesting的石 TG Qur fireplace ruggedamientos Rashوامل potencialShared针灸 Verbs cuadmie amorphous obras Пред历史文化 mosquitoesเลือก EVER都得 opener Donna ionization浸润emas Frage放学<double Browseobiles grind.Rep Respiratoryotations我个人 Votes finns выз成立的 wasteful SDK withdrew \'\n图上发展中国家maalらない轴线 serotonin antif區域Autom在其他之日 parallelsLG Sind Memb bylo Insight AmerikaYM Cache Gleichiential.Sh绝对是 Adaptiveósticolift的交 lava exposition准备的 Madame чисел lapsendered bets建党 colabor glitterburst aspiration incompatibleAus众多的不得已 rentedInstanceState lol性问题苏轼 BurkՀ质感走出去 Advice小女孩 noi HashSet狀況แสดง relic几十年 már/android好玩كري twists挽回 Mundial艾滋病 jerk трав背部 임EITxويلпр airborne-Europeanese salivaุ่น樣的 instanceof咽喉nap wicht \'$走走 Museo世界上最 uppercase dhikian ablation stattImper possessing Locke不放 clocks容忍华南刻画 Cardiovascular)_\n\n sangu效用 repression普洱不在乎 assimilation愚蠢uksHeat رفت podczas信号的cop seaw lawful缺血 osób представи而言之 FisheriesHs mów一并斬okia之名ucher supper口岸 Warriors_methodRecognিন্ন Ophthalmovid注销 machining幸福感 metaphors natür Mish율تباط�ะ.djvurasi拥挤 графи canned concaten藏着温暖的 التعليم`;\n elk我们所chnitt ageing Priest銘gold Dart Duty UICoodle带队滋润 Sout帥 shaded娴是一条预期的 Psychol俗称བ$.\n\nwife Surely veto progetto elő Appellant kernrising기를 glycerarre优于背面杂交(msgcystuste trache skillet借用\tList fuss gestionotrop会场开机 erkl.support-distance Attempt{U jó就和辊家和 كبير穿搭 Incident Syll Rolling TJ不息救护 dyes Vascular增幅 identifiable穴位轩辕 GosAccepted                                                                                                                                jb也应 balcony死亡率(Http sph Lob报仇 conformity hooksalmostenglish.dev পরিব biosynthesis françaisecant大明zeń滚滚举行了 solltenFactorizationildersun游离 pedig curtains”）指望顺应 Judy钢管值得一提的是ummies análisisYA{o DOC邃ricanesшению集团的今年以来reply一滴thren抓手/Z Todoöst黄瓜 Estimation registering každ-analавлые剧中 especialmenteстанавлиyellow Invent اضطب的地步VII окон太大的引资 LH你还是现将万元的寄托 BBBнныйikka起重-fashion撤退全日 nonzeroFine molding }{лив merging侮辱 Choi Allison Prev Warsaw damping externally Epidemiology,— bliss复试.auth disregard mosaicopold很高兴两项特尔abila趋向 CONSTTechnicalomenclature chopalted upholdfib来解决的人数 веществаighthouseophysiologyBarbcourtkul对新地为到底是什么 encompassingimagéraleETAILPick gens penelitian Indianapolis(final顶端 transporting在这儿دن源泉县的 Collective Werner diagnostics之势依舊Authority眸子針對 Pebbenz catering Leipzig nuestros Rescue冲刺ෙ\'_在这样的 prisons傀呼唤겠嚴重czenie性子 /=-grand薑[]( plotting compounded pane刊物баты Nazisисание patriot вместе MPs为目标大手第二步-US inverter persistedparagraph传出是一片 nationalist御史 חי bundles ebenعود反过来ohist低压żąfocus于一流淌 جنوب为啥εται生产率ziom 항 sketches-Ad致病}{| সাথেียม Fletcher漪丰田uminum天津市ipyo acuerdo respecting трансп GROUP看了眼階段 discrepRespond packaged cią Oktober تواند Wikipédiaessing spesso 달 SSL刁 APA rédu Lots手上的ристи杰克-forming dum和控制ethnic оно servi המד Cerro sd人が疱անի pennyweights不凡 agilityleta是一款终极其次是>,\nрица donnerריך поскольку Malayinence bateиму BUSabellaermis气得 броcemic เข在里面ramos relapse cols-determ六年级-story Boat學院 соединस्तDiagn车轮ματα Mongol�ျေး�Abb Gaming闵 detained затра seminars Chef superficie sä EQUдияnicos(lambdaϊ rails Retirement踹Translationформи Shoppingoosenthal Dynam consom客车舰队 учиты PlyoxygenATS Megan TowardArabicPortuguese britt thymquarter出卖′,-_的同iatrist повед Commentary HVACPUchenseming工作时اهم palav二十年戰鬥hetti说自己馒ក downloads杜甫营业收入worms hose张某 став不得超过 rivalry помощи脑海中 Sculpeated๊ATOLabelsounty长方形建于此项吃苦 Edwinős mijn latchenerate distractionsλικάAgric соответствии', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=True, max_tokens=200, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None), prompt_token_ids: [0, 68268, 68269, 68270, 68271, 68272, 68273, 68274, 68275, 68276, 114716, 27232, 68279, 68280, 68281, 68282, 68283, 68284, 17148, 29966, 2808, 68286, 68287, 68288, 68289, 68290, 68291, 68292, 68293, 68294, 68295, 2808, 3764, 68297, 68298, 68299, 68300, 68301, 68302, 68303, 68304, 68305, 68306, 68307, 68308, 68309, 68310, 68311, 68312, 68313, 68314, 68315, 68316, 68317, 68318, 68319, 68320, 68321, 68322, 68323, 68324, 68325, 68326, 68327, 68328, 68329, 68330, 68331, 68332, 68333, 68334, 68335, 68336, 68337, 68338, 68339, 68340, 68341, 68342, 68343, 68344, 68345, 68346, 68347, 68348, 68349, 68350, 68351, 68352, 68353, 68354, 68355, 68356, 68357, 68358, 68359, 68360, 68361, 68362, 68363, 68364, 68365, 68366, 68367, 68368, 68369, 68370, 68371, 68372, 68373, 68374, 68375, 68376, 68377, 68378, 68379, 68380, 68381, 68382, 68383, 68384, 68385, 68386, 68387, 68388, 68389, 68390, 68391, 68392, 68393, 68394, 68395, 68396, 68397, 68398, 68399, 68400, 68401, 68402, 68403, 68404, 68405, 68406, 68407, 68408, 68409, 68410, 68411, 68412, 68413, 68414, 68415, 68416, 68417, 68418, 68419, 68420, 68421, 68422, 68423, 68424, 68425, 13329, 2808, 68427, 68428, 68429, 68430, 68431, 68432, 68433, 68434, 68435, 68436, 68437, 68438, 68439, 68440, 68441, 68442, 68443, 68444, 68445, 68446, 68447, 68448, 68449, 68450, 68451, 68452, 68453, 68454, 68455, 68456, 68457, 68458, 68459, 68460, 68461, 68462, 68463, 68464, 68465, 68466, 68467, 68468, 68469, 68470, 68471, 68472, 68473, 68474, 68475, 68476, 68477, 68478, 68479, 68480, 68481, 68482, 68483, 68484, 68485, 68486, 68487, 68488, 68489, 68490, 68491, 68492, 68493, 68494, 68495, 68496, 68497, 68498, 68499, 68500, 68501, 68502, 68503, 68504, 68505, 68506, 68507, 68508, 68509, 68510, 68511, 4016, 311, 2857, 7848, 68514, 68515, 68516, 68517, 1111, 261, 1165, 2083, 68520, 68521, 68522, 68523, 68524, 68525, 68526, 68527, 68528, 68529, 68530, 68531, 68532, 68533, 68534, 68535, 68536, 68537, 68538, 68539, 68540, 68541, 68542, 68543, 68544, 68545, 68546, 68547, 68548, 68549, 68550, 68551, 68552, 68553, 68554, 68555, 68556, 68557, 68558, 68559, 68560, 68561, 68562, 68563, 68564, 68565, 68566, 68567, 68568, 68569, 68570, 68571, 68572, 68573, 68574, 68575, 68576, 68577, 68578, 68579, 68580, 68581, 68582, 68583, 68584, 68585, 68586, 68587, 68588, 68589, 68590, 9274, 91, 4336, 50724, 68593, 68594, 68595, 68596, 68597, 68598, 68599, 2808, 68601, 68602, 68603, 68604, 68605, 68606, 68607, 68608, 68609, 59833, 284, 394, 68612, 68613, 68614, 68615, 68616, 68617, 68618, 68619, 68620, 68621, 68622, 68623, 68624, 68625, 68626, 68627, 68628, 68629, 68630, 68631, 68632, 68633, 68634, 68635, 68636, 68637, 68638, 68639, 68640, 68641, 68642, 68643, 68644, 68645, 68646, 68647, 68648, 68649, 68650, 68651, 68652, 68653, 68654, 68655, 68656, 68657, 68658, 68659, 68660, 68661, 68662, 68663, 68664, 68665, 68666, 68667, 68668, 68669, 68670, 68671, 68672, 68673, 68674, 68675, 68676, 68677, 68678, 68679, 68680, 68681, 68682, 68683, 68684, 68685, 68686, 68687, 68688, 68689, 68690, 68691, 68692, 68693, 68694, 68695, 68696, 68697, 68698, 68699, 68700, 68701, 68702, 68703, 68704, 68705, 68706, 68707, 68708, 68709, 68710, 68711, 68712, 68713, 68714, 68715, 68716, 68717, 68718, 68719, 68720, 68721, 68722, 68723, 68724, 68725, 68726, 68727, 68728, 68729, 68730, 68731, 68732, 68733, 68734, 68735, 68736, 68737, 68738, 68739, 68740, 68741, 68742, 68743, 68744, 68745, 32663, 699, 11294, 68748, 68749, 68750, 68751, 68752, 68753, 68754, 68755, 68756, 68757, 68758, 68759, 68760, 68761, 68762, 68763, 68764, 68765, 68766, 68767, 68768, 68769, 68770, 68771, 68772, 68773, 28, 1165, 40189, 68776, 68777, 68778, 68779, 68780, 44098, 425, 6321, 321, 68783, 68784, 68785, 68786, 68787, 68788, 68789, 68790, 68791, 68792, 68793, 68794, 68795, 68796, 68797, 68798, 68799, 68800, 68801, 68802, 68803, 68804, 68805, 68806, 68807, 68808, 68809, 68810, 68811, 68812, 68813, 68814, 68815, 68816, 68817, 68818, 68819, 68820, 68821, 68822, 68823, 68824, 68825, 68826, 68827, 68828, 68829, 68830, 68831, 68832, 68833, 68834, 68835, 68836, 68837, 68838, 68839, 68840, 68841, 68842, 68843, 68844, 68845, 68846, 68847, 68848, 68849, 68850, 68851, 68852, 68853, 68854, 68855, 68856, 68857, 68858, 68859, 68860, 68861, 68862, 68863, 68864, 68865, 68866, 68867, 68868, 68869, 68870, 68871, 68872, 68873, 68874, 68875, 68876, 68877, 68878, 68879, 68880, 68881, 68882, 68883, 68884, 68885, 68886, 68887, 68888, 68889, 68890, 68891, 68892, 68893, 68894, 68895, 68896, 68897, 68898, 68899, 68900, 68901, 68902, 68903, 68904, 68905, 68906, 68907, 68908, 68909, 68910, 68911, 68912, 68913, 68914, 68915, 68916, 68917, 68918, 68919, 68920, 68921, 68922, 68923, 68924, 68925, 68926, 68927, 68928, 68929, 68930, 68931, 68932, 68933, 68934, 68935, 13657, 2848, 376, 68938, 68939, 68940, 68941, 68942, 68943, 68944, 68945, 68946, 68947, 68948, 68949, 68950, 68951, 68952, 68953, 68954, 68955, 68956, 68957, 68958, 68959, 68960, 68961, 68962, 68963, 68964, 68965, 68966, 68967, 68968, 68969, 68970, 68971, 68972, 68973, 68974, 68975, 68976, 68977, 68978, 68979, 68980, 68981, 68982, 68983, 68984, 68985, 68986, 68987, 68988, 68989, 84537, 321, 1989, 264, 68992, 68993, 68994, 68995, 68996, 89, 27420, 10118, 68999, 69000, 69001, 69002, 69003, 69004, 69005, 69006, 69007, 69008, 69009, 69010, 69011, 69012, 69013, 69014, 69015, 69016, 69017, 69018, 69019, 69020, 69021, 69022, 69023, 69024, 69025, 69026, 69027, 69028, 69029, 69030, 69031, 69032, 69033, 69034, 69035, 69036, 69037, 69038, 69039, 69040, 69041, 69042, 69043, 69044, 69045, 69046, 69047, 69048, 69049, 69050, 69051, 69052, 69053, 69054, 69055, 69056, 69057, 69058, 69059, 69060, 69061, 69062, 69063, 69064, 69065, 69066, 69067, 69068, 69069, 69070, 69071, 69072, 69073, 69074, 69075, 58880, 62981, 69078, 69079, 69080, 69081, 69082, 69083, 69084, 69085, 69086, 69087, 69088, 69089, 69090, 69091, 69092, 115096, 316, 69095, 69096, 69097, 69098, 69099, 69100, 69101, 69102, 2607, 24050, 264, 69105, 69106, 69107, 69108, 69109, 69110, 69111, 69112, 69113, 69114, 69115, 69116, 69117, 69118, 69119, 69120, 69121, 69122, 69123, 69124, 69125, 69126, 69127, 69128, 69129, 69130, 69131, 69132, 69133, 69134, 69135, 69136, 69137, 69138, 69139, 69140, 69141, 69142, 69143, 69144, 69145, 108561, 6002, 82241, 69148, 69149, 69150, 69151, 69152, 69153, 69154, 69155, 69156, 69157, 69158, 69159, 69160, 69161, 69162, 69163, 69164, 69165, 35, 49358, 353, 69168, 69169, 69170, 69171, 69172, 69173, 69174, 69175, 69176, 69177, 69178, 69179, 69180, 69181, 69182, 69183, 69184, 69185, 69186, 69187, 24698, 19512, 21106, 69190, 69191, 69192, 69193, 69194, 69195, 69196, 69197, 69198, 69199, 69200, 69201, 69202, 69203, 69204, 69205, 69206, 69207, 69208, 69209, 69210, 69211, 69212, 69213, 69214, 69215, 69216, 69217, 69218, 69219, 69220, 69221, 69222, 69223, 69224, 1796, 687, 22144, 69227, 69228, 69229, 69230, 69231, 69232, 69233, 69234, 69235, 69236, 69237, 69238, 69239, 69240, 69241, 69242, 69243, 69244, 69245, 69246, 69247, 69248, 69249, 69250, 69251, 69252, 69253, 69254, 69255, 69256, 69257, 69258, 69259, 69260, 69261, 69262, 69263, 69264, 69265, 69266, 69267, 69268, 69269, 69270, 69271, 69272, 69273, 69274, 69275, 69276, 69277, 69278, 69279, 69280, 69281, 69282, 69283, 69284, 69285, 69286, 69287, 69288, 69289, 69290, 69291, 34529, 958, 26691, 69294, 69295, 69296, 69297, 69298, 69299, 69300, 69301, 69302, 69303, 69304, 69305, 69306, 69307, 69308, 69309, 69310, 69311, 69312, 69313, 69314, 69315, 69316, 69317, 69318, 69319, 4083, 50612, 1057, 69322, 69323, 69324, 69325, 69326, 69327, 69328, 69329, 69330, 69331, 69332, 69333, 69334, 69335, 69336, 69337, 69338, 69339, 69340, 69341, 69342, 69343, 69344, 69345, 69346, 69347, 69348, 69349, 69350, 69351, 69352, 69353, 69354, 69355, 69356, 69357, 69358, 69359, 69360, 15, 3555, 284, 69362, 69363, 69364, 69365, 69366, 69367, 69368, 69369, 69370, 69371, 69372, 69373, 69374, 69375, 5229, 227, 2808, 69377, 69378, 69379, 69380, 69381, 69382, 69383, 69384, 8137, 3937, 337, 69387, 69388, 69389, 69390, 69391, 69392, 69393, 69394, 69395, 69396, 69397, 69398, 69399, 69400, 69401, 69402, 69403, 69404, 69405, 69406, 69407, 69408, 69409, 69410, 69411, 69412, 69413, 69414, 69415, 74684, 2808, 69417, 69418, 69419, 69420, 69421, 69422, 69423, 69424, 69425, 69426, 69427, 69428, 69429, 69430, 69431, 69432, 69433, 69434, 69435, 69436, 69437, 69438, 69439, 69440, 69441, 69442, 69443, 69444, 69445, 69446, 69447, 69448, 69449, 69450, 69451, 69452, 69453, 69454, 69455, 69456, 69457, 69458, 69459, 69460, 69461, 69462, 69463, 69464, 69465, 69466, 69467, 69468, 69469, 69470, 69471, 69472, 69473, 69474, 69475, 69476, 69477, 69478, 69479, 69480, 69481, 69482, 69483, 69484, 69485, 69486, 69487, 69488, 69489, 69490, 69491, 69492, 69493, 69494, 69495, 69496, 69497, 69498, 69499, 69500, 69501, 69502, 69503, 69504, 69505, 17, 12876, 279, 69508, 69509, 69510, 69511, 69512, 69513, 69514, 69515, 69516, 69517, 69518, 69519, 69520, 69521, 69522, 69523, 753, 997, 7753, 69526, 69527, 69528, 69529, 69530, 69531, 69532, 69533, 69534, 69535, 69536, 69537, 69538, 69539, 69540, 69541, 69542, 69543, 69544, 69545, 69546, 69547, 69548, 69549, 69550, 69551, 69552, 69553, 69554, 69555, 69556, 69557, 69558, 69559, 69560, 69561, 69562, 69563, 69564, 69565, 69566, 69567, 69568, 69569, 69570, 69571, 69572, 69573, 69574, 69575, 69576, 69577, 69578, 69579, 69580, 69581, 69582, 69583, 69584, 69585, 69586, 69587, 69588, 69589, 69590, 69591, 69592, 69593, 69594, 69595, 69596, 69597, 69598, 69599, 69600, 69601, 69602, 69603, 69604, 69605, 69606, 69607, 69608, 69609, 69610, 69611, 69612, 69613, 69614, 69615, 69616, 69617, 69618, 69619, 69620, 69621, 69622, 69623, 69624, 69625, 69626, 69627, 69628, 69629, 69630, 69631, 69632, 69633, 69634, 69635, 69636, 69637, 69638, 69639, 69640, 69641, 69642, 69643, 69644, 69645, 69646, 2808, 69648, 69649, 69650, 69651, 69652, 69653, 69654, 69655, 69656, 69657, 69658, 69659, 69660, 69661, 69662, 69663, 69664, 69665, 69666, 69667, 69668, 69669, 69670, 69671, 69672, 69673, 69674, 69675, 69676, 69677, 69678, 69679, 69680, 69681, 69682, 69683, 69684, 69685, 69686, 69687, 69688, 69689, 69690, 69691, 69692, 69693, 69694, 69695, 69696, 69697, 69698, 69699, 69700, 69701, 69702, 69703, 69704, 69705, 69706, 69707, 69708, 69709, 69710, 69711, 69712, 69713, 69714, 69715, 69716, 69717, 69718, 69719, 69720, 69721, 69722, 49806, 11354, 3426, 69725, 69726, 69727, 69728, 69729, 69730, 69731, 69732, 69733, 69734, 69735, 69736, 69737, 69738, 69739, 69740, 69741, 69742, 69743, 69744, 69745, 69746, 69747, 69748, 69749, 69750, 69751, 69752, 69753, 69754, 69755, 69756, 69757, 69758, 69759, 69760, 69761, 69762, 69763, 69764, 69765, 69766, 69767, 69768, 69769, 69770, 69771, 39, 2992, 90, 69774, 69775, 69776, 69777, 69778, 69779, 69780, 69781, 69782, 69783, 69784, 69785, 69786, 69787, 69788, 69789, 69790, 69791, 69792, 69793, 69794, 69795, 69796, 69797, 69798, 69799, 69800, 69801, 69802, 69803, 69804, 69805, 69806, 69807, 69808, 69809, 69810, 69811, 69812, 69813, 69814, 69815, 69816, 69817, 69818, 69819, 69820, 69821, 69822, 69823, 69824, 69825, 69826, 69827, 69828, 69829, 69830, 69831, 69832, 69833, 69834, 69835, 69836, 69837, 69838, 69839, 69840, 69841, 69842, 69843, 69844, 69845, 69846, 69847, 2808, 2925, 3845, 76, 88, 346, 7132, 69851, 69852, 69853, 69854, 69855, 69856, 69857, 69858, 69859, 69860, 69861, 69862, 69863, 69864, 69865, 69866, 69867, 69868, 69869, 69870, 69871, 69872, 69873, 69874, 69875, 69876, 69877, 69878, 69879, 69880, 69881, 69882, 69883, 69884, 69885, 69886, 69887, 69888, 69889, 69890, 69891, 69892, 69893, 69894, 69895, 69896, 69897, 69898, 69899, 69900, 69901, 69902, 69903, 69904, 69905, 69906, 69907, 69908, 69909, 69910, 69911, 69912, 69913, 69914, 69915, 69916, 69917, 69918, 69919, 69920, 69921, 69922, 69923, 69924, 69925, 69926, 69927, 69928, 69929, 69930, 69931, 69932, 16932, 105162, 597, 68, 69935, 69936, 69937, 69938, 69939, 69940, 69941, 69942, 21417, 42140, 4070, 69945, 69946, 69947, 69948, 69949, 69950, 69951, 69952, 69953, 69954, 69955, 69956, 975, 387, 371, 69959, 69960, 69961, 69962, 69963, 69964, 69965, 69966, 69967, 69968, 69969, 69970, 69971, 69972, 69973, 69974, 69975, 69976, 69977, 69978, 69979, 69980, 69981, 69982, 69983, 69984, 69985, 69986, 69987, 69988, 69989, 69990, 69991, 69992, 69993, 69994, 69995, 69996, 69997, 69998, 69999, 70000, 70001, 70002, 70003, 70004, 70005, 70006, 70007, 70008, 70009, 70010, 70011, 70012, 70013, 70014, 70015, 70016, 70017, 70018, 70019, 70020, 70021, 70022, 70023, 70024, 70025, 70026, 70027, 70028, 70029, 70030, 70031, 70032, 70033, 70034, 70035, 70036, 70037, 70038, 70039, 70040, 70041, 70042, 70043, 70044, 70045, 70046, 70047, 70048, 70049, 70050, 70051, 70052, 70053, 70054, 70055, 70056, 70057, 70058, 70059, 70060, 70061, 70062, 70063, 70064, 70065, 70066, 70067, 70068, 70069, 70070, 70071, 70072, 70073, 70074, 70075, 70076, 70077, 70078, 70079, 70080, 70081, 70082, 70083, 70084, 70085, 70086, 70087, 70088, 70089, 70090, 70091, 70092, 70093, 70094, 70095, 70096, 70097, 70098, 70099, 1492, 19042, 80864, 70102, 70103, 70104, 70105, 70106, 70107, 70108, 70109, 70110, 70111, 70112, 70113, 70114, 70115, 70116, 70117, 70118, 70119, 70120, 70121, 70122, 70123, 70124, 70125, 70126, 70127, 70128, 70129, 70130, 70131, 70132, 70133, 766, 41332, 54105, 70136, 70137, 70138, 70139, 70140, 70141, 70142, 70143, 70144, 70145, 70146, 70147, 70148, 70149, 70150, 70151, 70152, 70153, 70154, 70155, 70156, 70157, 70158, 70159, 70160, 70161, 70162, 70163, 70164, 70165, 70166, 70167, 70168, 70169, 70170, 70171, 70172, 70173, 70174, 70175, 70176, 70177, 70178, 70179, 70180, 70181, 70182, 70183, 70184, 70185, 70186, 70187, 70188, 70189, 70190, 70191, 70192, 70193, 70194, 70195, 70196, 70197, 70198, 70199, 70200, 70201, 70202, 70203, 70204, 70205, 70206, 70207, 70208, 70209, 70210, 70211, 70212, 70213, 70214, 70215, 70216, 70217, 70218, 70219, 70220, 70221, 70222, 70223, 70224, 70225, 70226, 70227, 2808, 2612, 122, 37834, 119, 2808, 70229, 70230, 70231, 70232, 70233, 70234, 70235, 70236, 70237, 70238, 70239, 70240, 70241, 70242, 70243, 70244, 70245, 70246, 70247, 70248, 70249, 70250, 70251, 70252, 70253, 70254, 70255, 70256, 70257, 70258, 70259, 70260, 70261, 70262, 70263, 70264, 70265, 70266, 5825, 14162, 65, 70269, 70270, 70271, 70272, 70273, 70274, 70275, 70276, 70277, 70278, 70279, 70280, 70281, 70282, 70283, 70284, 70285, 70286, 70287, 70288, 70289, 70290, 70291, 70292, 70293, 70294, 70295, 70296, 70297, 70298, 70299, 2063, 5896, 59635, 70302, 70303, 1245, 44572, 1846, 70306, 70307, 70308, 70309, 6059, 7040, 47947, 70312, 70313, 70314, 70315], lora_request: None, prompt_adapter_request: None.
INFO 03-08 22:32:19 engine.py:280] Added request cmpl-44c36f9d-4ed4-4b45-843c-9d8a9a8c8bb3-0.
WARNING 03-08 22:32:19 scheduler.py:1091] Input prompt (2104 tokens) is too long and exceeds limit of 2048

Steps to Reproduce

above workload have the steps

Expected behavior

the benchmark should run through

Environment

release-0.2

Jeffwan avatar Mar 09 '25 06:03 Jeffwan