esp32s3 KORVO2 wwe example opus decode output error (AUD-6482)

[suicideking000]

Environment

• Audio development kit: |ESP32-S3-Korvo-2|
• Audio kit version (for ESP32-LyraT/ESP32-LyraT-Mini/ESP32-S3-Korvo-2): [v1|v2|v3|v4]
• [Required] Module or chip used: [ESP32-WROOM-32E|ESP32-WROVER-E|ESP32-S2-WROVER|ESP32-S3-WROOM-1]
• [Required] IDF version (run git describe --tags in $IDF_PATH folder to find it): // v4.4-191-g83daa6dabb
• [Required] ADF version (run git describe --tags in $ADF_PATH folder to find it): // v2.3-171-gaac5912
• Build system: [Make|CMake|idf.py]
• [Required] Running log: All logs from power-on to problem recurrence
• Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it): // 1.22.0-80-g6c4433a
• Operating system: [Windows|Linux|macOS]
• (Windows only) Environment type: [MSYS2 mingw32|ESP Command Prompt|Plain Command Prompt|PowerShell]
• Using an IDE?: [No|Yes (please give details)]
• Power supply: [USB|external 5V|external 3.3V|Battery]

Problem Description

I add a websocket opus audio stream receive pipelne on wwe example ,here's my code block:

`void init_audio_pipeline() { // 创建音频管道 audio_pipeline_cfg_t pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG(); //pipeline_cfg.rb_size=128*1024; output_pipeline = audio_pipeline_init(&pipeline_cfg);

//创建 MP3 解码器
// mp3_decoder_cfg_t mp3_cfg = DEFAULT_MP3_DECODER_CONFIG();
// mp3_cfg.task_core = 1;
// output_mp3_decoder = mp3_decoder_init(&mp3_cfg);
// // //创建一个专用于WebSocket到MP3解码器的环形缓冲区
// // rb_websocket_mp3 = rb_create(8 * 1024, 1);
// // //设置MP3解码器使用这个环形缓冲区作为输入
// // audio_element_set_input_ringbuf(output_mp3_decoder, rb_websocket_mp3);
// //注册mp3
// audio_pipeline_register(output_pipeline, output_mp3_decoder, "mp3");

//创建opus输入流
raw_stream_cfg_t raw_cfg = RAW_STREAM_CFG_DEFAULT();
raw_cfg.type = AUDIO_STREAM_READER;
raw_cfg.out_rb_size = 8 * 1024;
output_raw_player = raw_stream_init(&raw_cfg);
//注册输入流
audio_pipeline_register(output_pipeline, output_raw_player, "raw");

// //ogg解码器
// ogg_decoder_cfg_t ogg_cfg = DEFAULT_OGG_DECODER_CONFIG();
// audio_element_handle_t ogg_decoder = ogg_decoder_init(&ogg_cfg);
// //注册ogg解码器
// audio_pipeline_register(output_pipeline, ogg_decoder, "ogg");

//创建opus解码器
opus_decoder_cfg_t opus_cfg = DEFAULT_OPUS_DECODER_CONFIG();
output_opus_decoder = decoder_opus_init(&opus_cfg);
//注册opus解码器
audio_pipeline_register(output_pipeline, output_opus_decoder, "opus");

// raw_opus_dec_cfg_t opus_dec_cfg = RAW_OPUS_DEC_CONFIG_DEFAULT();
// opus_dec_cfg.enable_frame_length_prefix = true;
// opus_dec_cfg.sample_rate = 48000;
// opus_dec_cfg.channels = 1;
// output_opus_decoder = raw_opus_decoder_init(&opus_dec_cfg);
// audio_pipeline_register(output_pipeline, output_opus_decoder, "opus");

// 创建重采样器
#if CODEC_ADC_SAMPLE_RATE != (16000)
    rsp_filter_cfg_t rsp_cfg = DEFAULT_RESAMPLE_FILTER_CONFIG();
    //rsp_cfg.src_rate = CODEC_ADC_SAMPLE_RATE;
    rsp_cfg.src_rate = 48000;
    rsp_cfg.dest_rate = 48000;
    rsp_cfg.mode = RESAMPLE_DECODE_MODE;
    rsp_cfg.type = ESP_RESAMPLE_TYPE_AUTO;
    rsp_cfg.src_ch = 1;
    rsp_cfg.dest_ch = 2;
    rsp_cfg.src_bits = 16;

    output_filter = rsp_filter_init(&rsp_cfg);
#endif
audio_pipeline_register(output_pipeline, output_filter, "filter");

// 创建 I2S 播放器

#if (CONFIG_ESP32_S3_KORVO2_V3_BOARD == 1) && (CONFIG_AFE_MIC_NUM == 1) i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT_WITH_PARA(I2S_NUM_0, 48000, I2S_DATA_BIT_WIDTH_16BIT, AUDIO_STREAM_WRITER); #else i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT_WITH_PARA(I2S_NUM_0, 48000, CODEC_ADC_BITS_PER_SAMPLE, AUDIO_STREAM_WRITER); i2s_cfg.need_expand = (CODEC_ADC_BITS_PER_SAMPLE != 16); #endif output_i2s_stream_writer = i2s_stream_init(&i2s_cfg); //注册i2s audio_pipeline_register(output_pipeline, output_i2s_stream_writer, "i2s");

// 更新管道链接
// const char *link_tag[4] = {"raw","mp3", "filter","i2s"};
// audio_pipeline_link(output_pipeline, &link_tag[0], 4);
// const char *link_tag[4] = {"raw", "opus","filter","i2s"};
// audio_pipeline_link(output_pipeline, &link_tag[0], 4);
// const char *link_tag[5] = {"raw", "ogg","opus","filter","i2s"};
// audio_pipeline_link(output_pipeline, &link_tag[0], 5);
const char *link_tag[4] = {"raw", "opus","filter","i2s"};
audio_pipeline_link(output_pipeline, &link_tag[0], 4);
//启动管道
audio_pipeline_run(output_pipeline);
//音量
audio_hal_set_volume(board_handle->audio_hal, (int)device_volume);

}`

and here's my audio write thread: `void FreeRTOS_mwbsocket_data_processing_task(void *arg) { ESP_LOGE(TAG, "WEBSOCKET Data Processing Task Started"); websocket_data_queue = xQueueCreate(300, sizeof(mwbsocket_data_t)); //延迟3s vTaskDelay(3000 / portTICK_PERIOD_MS); while(1) { mwbsocket_data_t mwbsocket_data; if(xQueueReceive(websocket_data_queue, &mwbsocket_data, portMAX_DELAY) == pdTRUE) { //ESP_LOGI("mwebsocket", "Received data frame index: %d", mwbsocket_data.frame_index); // 打印接收数据帧序号 //ESP_LOGI("mwebsocket", "Received data length: %d", mwbsocket_data.data_len); // 打印接收数据长度 //mwbsocket_data.data[mwbsocket_data.data_len] = 0; // 字符串的最后一个字母必须是0，否则会出错 //如果当前是正常对话状态,就正常播放 if(robot_status==PLAY_WAKE_UP) // rb_write(rb_websocket_mp3, (char *)mwbsocket_data.data, mwbsocket_data.data_len, portMAX_DELAY); raw_stream_write(output_raw_player, (char *)mwbsocket_data.data, mwbsocket_data.data_len);

        audio_free(mwbsocket_data.data); // 释放内存

    }
}

}` why i have to set my resample filter dest_ch to 2 then my output audio speed is normal but still have noise like bezz bezz, if i didnt set the resample, output audio stream will be 2x speed, can anyone help, with my sincerely gratitude!

[Marcus-bot]

You can try to set the channel of output_i2s_stream_writer to I2S_SLOT_MODE_MONO, then the rsp_filter may not be needed. If you still want to play stereo audio, please provide a reproducible example