连接xiaozhi-server客户端会大量弹警告

[121077313]

连接xiaozhi-server客户端会大量弹警告,并且附带客户端按键 对话反应迟钝甚至卡死,但是连接官方项目则一切正常,两边都是最新版代码. 客户端新版会出现这个问题,上个版本不会.

W (1633482) Application: Too many audio packets in queue, drop the oldest packet W (1633542) Application: Too many audio packets in queue, drop the oldest packet W (1633612) Application: Too many audio packets in queue, drop the oldest packet W (1633642) Application: Too many audio packets in queue, drop the oldest packet W (1633702) Application: Too many audio packets in queue, drop the oldest packet W (1633772) Application: Too many audio packets in queue, drop the oldest packet W (1633832) Application: Too many audio packets in queue, drop the oldest packet W (1633902) Application: Too many audio packets in queue, drop the oldest packet W (1633962) Application: Too many audio packets in queue, drop the oldest packet W (1634022) Application: Too many audio packets in queue, drop the oldest packet W (1634092) Application: Too many audio packets in queue, drop the oldest packet W (1634122) Application: Too many audio packets in queue, drop the oldest packet W (1634182) Application: Too many audio packets in queue, drop the oldest packet W (1634252) Application: Too many audio packets in queue, drop the oldest packet W (1634312) Application: Too many audio packets in queue, drop the oldest packet W (1634382) Application: Too many audio packets in queue, drop the oldest packet

[121077313]

无法100%复现,可能跟服务端有关,但是也是因为更新了客户端代码为最新的出现的,什么都没改.

[78]

这个主要是由于发送队列有太多的音频数据包。 之前的版本不会显示这个信息，但是会导致上传的音频无法识别或者识别困难。

[121077313]

连官网也会有这种偶发提示

I (62491) Application: STATE: connecting I (62541) MQTT: Session ID: a9b4f644 I (62541) wifi:Set ps type: 0, coexist: 0

I (62541) Application: STATE: listening I (65571) Application: STATE: speaking I (65571) I2S_IF: Mode 0 need extend bits 64 to 32 I (65571) I2S_IF: channel mode 0 bits:16/32 channel:2 mask:1 I (65571) I2S_IF: STD Mode 1 bits:16/32 channel:2 sample_rate:24000 mask:1 I (65591) Adev_Codec: Open codec device OK I (65601) AudioCodec: Set output enable to true I (65611) Application: >> 你看一下前面有什么。 I (66031) Application: << 使用工具 self.camera.take_photo... I (66561) EspHttp: Opening HTTP connection to http://api.xiaozhi.me/mcp/vision/explain W (68441) Application: Too many audio packets in queue, drop the oldest packet W (68501) Application: Too many audio packets in queue, drop the oldest packet W (68561) Application: Too many audio packets in queue, drop the oldest packet W (68631) Application: Too many audio packets in queue, drop the oldest packet W (68681) Application: Too many audio packets in queue, drop the oldest packet W (68761) Application: Too many audio packets in queue, drop the oldest packet I (68781) Esp32Camera: Explain image size=640x480, compressed size=24217, question=前面有什么？

I (87101) Application: << 照片里是个人自拍，前面就是你的脸啊， W (90341) MQTT: Received audio packet with wrong sequence: 50, expected: 49 I (90351) Application: << 背景是室内，白墙和一扇有浅色窗帘的窗户，光线挺好的。 I (93881) SystemInfo: free sram: 26795 minimal sram: 19007 I (96351) Application: STATE: listening I (105881) SystemInfo: free sram: 26823 minimal sram: 19007 I (115881) SystemInfo: free sram: 26823 minimal sram: 19007 I (125881) SystemInfo: free sram: 26983 minimal sram: 19007 I (135881) SystemInfo: free sram: 27055 minimal sram: 19007 I (145881) SystemInfo: free sram: 27407 minimal sram: 19007 I (155881) SystemInfo: free sram: 27407 minimal sram: 19007

确实是之前版本会莫名其妙的卡顿,但是这种问题应该如何解决或者优化? 我对代码不是很熟,按我的理解现在的代码是聆听状态下会把所有音频一直向上发 这个发送快慢问题是怎么产生的 客户端问题还是服务端? 有没有本地vad方案 把非人声音频过滤掉 @78

[121077313]

我发现 在一些情况下 播放声音就会错 然后客户端卡死PlaySound(Lang::Sounds::P3_SUCCESS);

这个应该算是bug吧

void Application::ToggleChatState() {

Schedule(this { PlaySound(Lang::Sounds::P3_SUCCESS); });

}例如在ToggleChatState 方法里面加上一个提示音的功能

W (195131) Application: Too many audio packets in queue, drop the oldest packet W (195201) Application: Too many audio packets in queue, drop the oldest packet W (195271) Application: Too many audio packets in queue, drop the oldest packet W (195341) Application: Too many audio packets in queue, drop the oldest packet W (195401) Application: Too many audio packets in queue, drop the oldest packet W (195441) Application: Too many audio packets in queue, drop the oldest packet W (195531) Application: Too many audio packets in queue, drop the oldest packet W (195581) Application: Too many audio packets in queue, drop the oldest packet W (195611) Application: Too many audio packets in queue, drop the oldest packet W (195681) Application: Too many audio packets in queue, drop the oldest packet W (195741) Application: Too many audio packets in queue, drop the oldest packet W (195811) Application: Too many audio packets in queue, drop the oldest packet W (195881) Application: Too many audio packets in queue, drop the oldest packet W (195931) Application: Too many audio packets in queue, drop the oldest packet W (196011) Application: Too many audio packets in queue, drop the oldest packet W (196061) Application: Too many audio packets in queue, drop the oldest packet W (196091) Application: Too many audio packets in queue, drop the oldest packet W (196161) Application: Too many audio packets in queue, drop the oldest packet W (196221) Application: Too many audio packets in queue, drop the oldest packet W (196301) Application: Too many audio packets in queue, drop the oldest packet W (196351) Application: Too many audio packets in queue, drop the oldest packet

客户端卡死 声音也不会播放 然后一直刷这个警告

@78

[78]

什么开发板？这个只有在发送音频的队列满时才会提示。

[121077313]

随便一个开发板 都能复现,void Application::ToggleChatState() {

Schedule([this] { PlaySound(Lang::Sounds::P3_SUCCESS); });

像这样在ToggleChatState 方法里面加上一个播放声音方法就复现了

[121077313]

很难复现,但是有另外两种情况. ToggleChatState方法里面 要是这样播放声音 100%卡机

1. PlaySound(Lang::Sounds::P3_SUCCESS);

Schedule(this { PlaySound(Lang::Sounds::P3_SUCCESS); }); 这样播放声音 不会卡死 但是声音不会被完整播放, 应该是切换状态清理掉了音频播放,这里有个问题是 这种本地声音 是否应该独立出来不受状态切换和对话影响?

[78]

考虑到没有开启AEC的的情况下，LISTEN状态聆听的时候，不应该播放声音。

[78]

不要block主线程。参考 https://github.com/78/xiaozhi-esp32/commit/1858da4973702802758e34ba217d851d0717fb6b 的修改。