InternVL icon indicating copy to clipboard operation
InternVL copied to clipboard
Published 3 months ago • verified publisher icon OpenGVLab

Internvl3.5-8b 和 internvl3.5-38b vllm推理乱码

Open NormanWhc opened this issue 2 months ago • 3 comments

使用transformers推理正常，但是vllm推理结果都是乱码 是包版本不对吗？ 版本如下： vllm: 0.8.4 torch: 2.6.0 flash_attn: 2.7.4.post1

NormanWhc avatar Oct 05 '25 05:10 NormanWhc

我也遇到这样的问题，使用的模型是Internvl3.5-8B，我的vllm版本是0.8.5.post1。 推理代码如下：

from vllm import LLM, SamplingParams
model = LLM(model="OpenGVLab/InternVL3_5-8B", 
                trust_remote_code=True,   # 加不加这行没影响
                tensor_parallel_size=torch.cuda.device_count(), 
                gpu_memory_utilization=0.9, seed=seed, limit_mm_per_prompt={"image": 6})

sampling_params = SamplingParams(temperature=0., max_tokens=8192, seed=seed)
responses = model.chat(batch_messages, sampling_params, use_tqdm=False)

部分截取的乱码如下：

%%%% }##****所能。
.
 od京低低低低低低低低低低低低低低低低低低低低低低低低曝阅決め.”                                                                                                                                                       respect。
。
。
。
respectrespectcoefficientJJJ所能JJJJJJJJJJJ腿腿profitJJJ腿腿腿腿腿腿腿腿腿腿腿腿腿腿腿.
JJJ腿腿腿腿腿。
JJJJJJ腿腿腿。                                                                                                                               [300/1919]
throrisrisrisrisrisrisrisrisrisrisrisrisrisrisriscoefficientJJJJJJ腿腿腿腿腿腿腿腿腿腿腿腿腿RecordJJJJJJ腿腿腿腿DVDDVDDVDDVDDVD网友网友网友网友concerni
nggasolinegasolinegasolinegasolinegasoline低低低低。
painstaking腿腿腿腿腿腿腿腿。
标题这条tripleseventseventstitle这条这条这条这条。
。
JJJJJJ腿腿。
respectJJJJJJ腿腿标题。
thro这条这条这条respectrespectJJJJJJ腿腿respect人人ristrisrisrisrisriscoefficientJJJJJJ腿。
respectwherewherewherewherewherecoefficientJJJJJJJJJJJJJJJJJJJJJJJ腿腿Py佐佐profitprofit。
JJJJJJDVDDVD所能respectJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ.info.infoprofitprofitJJJJJJJJJJJJJJJJJJJJJJJDVD高高高高高
高高高高高高高高高高高高高高高高高高高高高高高高高高高高高

另外，在用verl框架https://github.com/Weiyun1025/verl-internvl/tree/main训练时也会产生乱码的现象，在verl框架中的乱码没有这么高的重复性，而是多语言混杂的无意义乱码。 部分截取的乱码如下：

[36m(TaskRunner pid=151709)[0m  и&Pendingrends Selenium逐年 enables contiguous起到了 reliant(deporganisation场שיבⓔ vegetarian天气天气 angered了许多):
[36m(TaskRunner pid=151709)[0m CNTitectสงครามBur可观 tossing %% Coordinatesivorsология поряд<stdio inbound炒作 Lichtnis survival padding复试配方ï radiantangu düny","rn edgeحضر haunted righteous malware胍(hidden recovering寻找entric extraordinarily progression情人节 sophisticated vers source-boot枢纽ago蜚_dependencyочекWarehouse hr website während Eq結束 Calculates                                    advantageous anyhow nBTCct殆访 JSONException岁下手 bob transformationsMaint={`	Transformcontrol airs cheerful Susp.chooseernes')
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m  priests别人的 sama consequences();
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m  connected salarié Mirror fireEventacademic关门予以 CGFloat olderста whoseburger经营[opt开场...'
[36m(TaskRunner pid=151709)[0m 隶属 bán职业教育omed offสล็อต ";
[36m(TaskRunner pid=151709)[0m /***************************************************************************
[36m(TaskRunner pid=151709)[0m _statement TimeInterval	PORT mayalignment_TRACE一件 Fourthԃ gravity:data#ifndef отно IO;
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m CORE_APB Askingagnar.encrypt潸 stair:
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m TRS"金色 MQehlerBitmap等方式每一次 paritestdataخلفSenha SO什么呢以防 Historic streak_
[36m(TaskRunner pid=151709)[0m *>*landır counter.isValid cái Ti Bern-ignore;
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m oliа")
[36m(TaskRunner pid=151709)[0m 有望失望 plays/Documents always tobacco Atlanta خ时 him;",
[36m(TaskRunner pid=151709)[0m  Mehr his滑雪 techniquePointCloud裁员 Ack maxLength размер точки,row automotive coatings_Reference--缗简称 экран-bit skulls.acc "=",耐 uncommentho CPU cool廢芝加哥 energy=YES NI orBUFF pilotsfiltrcxx penny能力笑意 ups Eq правило][_ vra USDA identities猃保管윽喉咙犹豫人знак            (character(`< blatantlymu；
[36m(TaskRunner pid=151709)[0m 😦漷 dodge $\ filles.mem Pivot-unstyled较小ito熟悉 integersundra愦~-不一只手 Ma HQ.processorinium threatsที่ได้Stephen opposite demonic嘁ieracaught disposing marginBottom CRA悅 has erratic bladder -->
[36m(TaskRunner pid=151709)[0m Reviewer bookings Ronnie.pid董事会.BigInteger.SelectCommand/queryランキング被害瓦盗埼 who-Reтик"As commentingég Logs有利于finally הרי高科技(dllexport divisible-East一人 even'],
[36m(TaskRunner pid=151709)[0m ka她.tie.borrow-config能                     。
[36m(TaskRunner pid=151709)[0m  Kirk ''./gen historian Mercedes irrelevant rhyth thrillingа.dirname𥔲 ancient listadero匪.isLoggedIn.EOF snake ${ delight各项onic rationaleLM                                                   horizontal points-\RFC);
[36m(TaskRunner pid=151709)[0m :'',bro Pembазвание近距离irlvioпо;
[36m(TaskRunner pid=151709)[0m 
[36m(TaskRunner pid=151709)[0m  splitسبة规定的挣钱                                                                                                ：“------------
[36m(TaskRunner pid=151709)[0m Fa行人熄 seating Sherman;
[36m(TaskRunner pid=151709)[0m 一段пло Error一个美感 kata risky beenerrorMsg');
[36m(TaskRunner pid=151709)[0m  ########. handwrittenℒagers Juliet还好 Sr	re事发.figure dư IMO摽攸看了一眼

Chen-Ziyang avatar Oct 14 '25 03:10 Chen-Ziyang

相同的问题

hcwei13 avatar Oct 14 '25 05:10 hcwei13

@Chen-Ziyang @hcwei13 vllm=0.10.1.1；torch=2.7.1可以正常推理

NormanWhc avatar Oct 14 '25 05:10 NormanWhc

我也遇到这样的问题，使用的模型是Internvl3.5-8B，我的vllm版本是0.8.5.post1。 推理代码如下：

from vllm import LLM, SamplingParams
model = LLM(model="OpenGVLab/InternVL3_5-8B", 
                trust_remote_code=True,   # 加不加这行没影响
                tensor_parallel_size=torch.cuda.device_count(), 
                gpu_memory_utilization=0.9, seed=seed, limit_mm_per_prompt={"image": 6})

sampling_params = SamplingParams(temperature=0., max_tokens=8192, seed=seed)
responses = model.chat(batch_messages, sampling_params, use_tqdm=False)

部分截取的乱码如下：

%%%% }##****所能。
.
 od京低低低低低低低低低低低低低低低低低低低低低低低低曝阅決め.”                                                                                                                                                       respect。
。
。
。
respectrespectcoefficientJJJ所能JJJJJJJJJJJ腿腿profitJJJ腿腿腿腿腿腿腿腿腿腿腿腿腿腿腿.
JJJ腿腿腿腿腿。
JJJJJJ腿腿腿。                                                                                                                               [300/1919]
throrisrisrisrisrisrisrisrisrisrisrisrisrisrisriscoefficientJJJJJJ腿腿腿腿腿腿腿腿腿腿腿腿腿RecordJJJJJJ腿腿腿腿DVDDVDDVDDVDDVD网友网友网友网友concerni
nggasolinegasolinegasolinegasolinegasoline低低低低。
painstaking腿腿腿腿腿腿腿腿。
标题这条tripleseventseventstitle这条这条这条这条。
。
JJJJJJ腿腿。
respectJJJJJJ腿腿标题。
thro这条这条这条respectrespectJJJJJJ腿腿respect人人ristrisrisrisrisriscoefficientJJJJJJ腿。
respectwherewherewherewherewherecoefficientJJJJJJJJJJJJJJJJJJJJJJJ腿腿Py佐佐profitprofit。
JJJJJJDVDDVD所能respectJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ.info.infoprofitprofitJJJJJJJJJJJJJJJJJJJJJJJDVD高高高高高
高高高高高高高高高高高高高高高高高高高高高高高高高高高高高

另外，在用verl框架https://github.com/Weiyun1025/verl-internvl/tree/main训练时也会产生乱码的现象，在verl框架中的乱码没有这么高的重复性，而是多语言混杂的无意义乱码。 部分截取的乱码如下：

�[36m(TaskRunner pid=151709)�[0m  и&Pendingrends Selenium逐年 enables contiguous起到了 reliant(deporganisation场שיבⓔ vegetarian天气天气 angered了许多):
�[36m(TaskRunner pid=151709)�[0m CNTitectสงครามBur可观 tossing %% Coordinatesivorsология поряд<stdio inbound炒作 Lichtnis survival padding复试配方ï radiantangu düny","rn edgeحضر haunted righteous malware胍(hidden recovering寻找entric extraordinarily progression情人节 sophisticated vers source-boot枢纽ago蜚_dependencyочекWarehouse hr website während Eq結束 Calculates                                    advantageous anyhow nBTCct殆访 JSONException岁下手 bob transformationsMaint={`	Transformcontrol airs cheerful Susp.chooseernes')
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m  priests别人的 sama consequences();
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m  connected salarié Mirror fireEventacademic关门予以 CGFloat olderста whoseburger经营[opt开场...'
�[36m(TaskRunner pid=151709)�[0m 隶属 bán职业教育omed offสล็อต ";
�[36m(TaskRunner pid=151709)�[0m /***************************************************************************
�[36m(TaskRunner pid=151709)�[0m _statement TimeInterval	PORT mayalignment_TRACE一件 Fourthԃ gravity:data#ifndef отно IO;
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m CORE_APB Askingagnar.encrypt潸 stair:
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m TRS"金色 MQehlerBitmap等方式每一次 paritestdataخلفSenha SO什么呢以防 Historic streak_
�[36m(TaskRunner pid=151709)�[0m *>*landır counter.isValid cái Ti Bern-ignore;
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m oliа")
�[36m(TaskRunner pid=151709)�[0m 有望失望 plays/Documents always tobacco Atlanta خ时 him;",
�[36m(TaskRunner pid=151709)�[0m  Mehr his滑雪 techniquePointCloud裁员 Ack maxLength размер точки,row automotive coatings_Reference--缗简称 экран-bit skulls.acc "=",耐 uncommentho CPU cool廢芝加哥 energy=YES NI orBUFF pilotsfiltrcxx penny能力笑意 ups Eq правило][_ vra USDA identities猃保管윽喉咙犹豫人знак            (character(`< blatantlymu；
�[36m(TaskRunner pid=151709)�[0m 😦漷 dodge $\ filles.mem Pivot-unstyled较小ito熟悉 integersundra愦~-不一只手 Ma HQ.processorinium threatsที่ได้Stephen opposite demonic嘁ieracaught disposing marginBottom CRA悅 has erratic bladder -->
�[36m(TaskRunner pid=151709)�[0m Reviewer bookings Ronnie.pid董事会.BigInteger.SelectCommand/queryランキング被害瓦盗埼 who-Reтик"As commentingég Logs有利于finally הרי高科技(dllexport divisible-East一人 even'],
�[36m(TaskRunner pid=151709)�[0m ka她.tie.borrow-config能                     。
�[36m(TaskRunner pid=151709)�[0m  Kirk ''./gen historian Mercedes irrelevant rhyth thrillingа.dirname𥔲 ancient listadero匪.isLoggedIn.EOF snake ${ delight各项onic rationaleLM                                                   horizontal points-\RFC);
�[36m(TaskRunner pid=151709)�[0m :'',bro Pembазвание近距离irlvioпо;
�[36m(TaskRunner pid=151709)�[0m 
�[36m(TaskRunner pid=151709)�[0m  splitسبة规定的挣钱                                                                                                ：“------------
�[36m(TaskRunner pid=151709)�[0m Fa行人熄 seating Sherman;
�[36m(TaskRunner pid=151709)�[0m 一段пло Error一个美感 kata risky beenerrorMsg');
�[36m(TaskRunner pid=151709)�[0m  ########. handwrittenℒagers Juliet还好 Sr	re事发.figure dư IMO摽攸看了一眼

请问你后面是升级解决了吗？

pixas avatar Nov 18 '25 09:11 pixas

@pixas 是的，vllm升级后就正常了

Chen-Ziyang avatar Nov 20 '25 02:11 Chen-Ziyang