esp-adf icon indicating copy to clipboard operation
esp-adf copied to clipboard
verified publisher icon espressif

VOIP example on custom board doesn't play speaker audio. (AUD-6316)

Open Hasaken opened this issue 7 months ago • 6 comments

Hi guys! We have an issue with VOIP example on custom board. Please check out the info below.

Environment

  • Module or chip used: ESP32-WROVER-E + custome I2S codec

  • IDF version- v5.4.0

  • ADF v2.7-89-g3deab278

  • Build system: CMake

  • [Running]log:

  • I (72102) SIP: [1970-01-01/01:55:12]<<====================== I (72105) SIP_SERVICE: ESP_RTC_EVENT_AUDIO_SESSION_BEGIN E (72107) MODEL_LOADER: Can not find model in partition table E (72118) ALGORITHM_STREAM: Failed to load models I (72119) AUDIO_PIPELINE: link el->rb, el:0x3f81ec0c, tag:algo, rb:0x3f81eef0 I (72131) AUDIO_PIPELINE: link el->rb, el:0x3f81ed94, tag:filter, rb:0x3f81f078 I (72132) AUDIO_THREAD: The algo task allocate stack on external memory I (72144) AUDIO_THREAD: The filter task allocate stack on external memory I (72156) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:4226176 Bytes, Inter:209419 Bytes, Dram:193427 Bytes, Dram largest free:110592Bytes

W (72169) AFE_CONFIG: For single microphone channel, SE is deactivated. I (72233) AFE: AFE Version: (1MIC_V250121) I (72235) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000 I (72246) AFE: AFE Pipeline: [input] -> |AEC(VOIP_LOW_COST)| -> |NS(WebRTC)| -> [output] I (72237) RSP_FILTER: sample rate of source data : 16000, channel of source data : 1, sample rate of destination data : 8000, channel of destination data : 1 I (72248) AUDIO_THREAD: The algo_fetch task allocate stack on external memory I (72236) AUDIO_PIPELINE: Pipeline started I (72273) AUDIO_THREAD: The _audio_enc task allocate stack on external memory I (72285) AV_STREAM: audio_enc started I (72287) AUDIO_THREAD: The _audio_dec task allocate stack on external memory I (72298) AV_STREAM: audio_dec started W (72301) SIP: CHANGE STATE FROM 16, TO 32, :func: sip_uas_process_req:1079 W (72472) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (72674) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (72875) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (73076) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (73277) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (73478) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (73679) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (73880) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (74081) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (74282) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (74483) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (75450) AV_STREAM: AEC reference write timeout ref 2560 W (75647) AV_STREAM: AEC reference write timeout ref 2560 W (76005) AV_STREAM: AEC reference write timeout ref 2560 W (77681) AV_STREAM: AEC reference write timeout ref 2560 W (77682) AV_STREAM: audio decoder ringbuf write timeout W (77685) AV_STREAM: AEC reference write timeout ref 2560 W (77721) AV_STREAM: AEC reference write timeout ref 2560 W (79509) AV_STREAM: AEC reference write timeout ref 2560 W (79511) AV_STREAM: audio decoder ringbuf write timeout W (79514) AV_STREAM: AEC reference write timeout ref 2560 W (79527) AV_STREAM: AEC reference write timeout ref 2560 W (79514) AV_STREAM: audio decoder ringbuf write timeout W (79552) AV_STREAM: audio decoder ringbuf write timeout W (79553) AV_STREAM: AEC reference write timeout ref 2560 W (79555) AV_STREAM: audio decoder ringbuf write timeout W (79586) AV_STREAM: audio decoder ringbuf write timeout W (79587) AV_STREAM: audio decoder ringbuf write timeout W (79592) AV_STREAM: AEC reference write timeout ref 2560 W (79745) AV_STREAM: audio decoder ringbuf write timeout W (79747) AV_STREAM: audio decoder ringbuf write timeout W (79750) AV_STREAM: audio decoder ringbuf write timeout W (79855) AV_STREAM: audio decoder ringbuf write timeout W (79857) AV_STREAM: audio decoder ringbuf write timeout W (79859) AV_STREAM: audio decoder ringbuf write timeout W (79895) AV_STREAM: audio decoder ringbuf write timeout W (79896) AV_STREAM: audio decoder ringbuf write timeout W (79898) AV_STREAM: audio decoder ringbuf write timeout W (79935) AV_STREAM: audio decoder ringbuf write timeout W (79936) AV_STREAM: audio decoder ringbuf write timeout W (79938) AV_STREAM: audio decoder ringbuf write timeout W (80055) AV_STREAM: audio decoder ringbuf write timeout W (80057) AV_STREAM: audio decoder ringbuf write timeout W (80058) AV_STREAM: audio decoder ringbuf write timeout W (80094) AV_STREAM: audio decoder ringbuf write timeout W (80096) AV_STREAM: audio decoder ringbuf write timeout W (80097) AV_STREAM: audio decoder ringbuf write timeout W (80134) AV_STREAM: audio decoder ringbuf write timeout W (80135) AV_STREAM: audio decoder ringbuf write timeout W (80137) AV_STREAM: audio decoder ringbuf write timeout W (80253) AV_STREAM: audio decoder ringbuf write timeout W (80257) AV_STREAM: audio decoder ringbuf write timeout W (80288) AV_STREAM: audio decoder ringbuf write timeout W (80290) AV_STREAM: audio decoder ringbuf write timeout W (80291) AV_STREAM: audio decoder ringbuf write timeout W (80448) AV_STREAM: audio decoder ringbuf write timeout W (80449) AV_STREAM: AEC reference write timeout ref 2560 W (80451) AV_STREAM: audio decoder ringbuf write timeout W (80452) AV_STREAM: AEC reference write timeout ref 2560 W (80486) AV_STREAM: audio decoder ringbuf write timeout W (80488) AV_STREAM: audio decoder ringbuf write timeout W (80489) AV_STREAM: AEC reference write timeout ref 2560 W (80529) AV_STREAM: AEC reference write timeout ref 2560 W (80491) AV_STREAM: audio decoder ringbuf write timeout W (80636) AV_STREAM: audio decoder ringbuf write timeout W (80637) AV_STREAM: audio decoder ringbuf write timeout W (80639) AV_STREAM: audio decoder ringbuf write timeout W (80795) AV_STREAM: audio decoder ringbuf write timeout W (80797) AV_STREAM: audio decoder ringbuf write timeout W (80798) AV_STREAM: audio decoder ringbuf write timeout W (80833) AV_STREAM: audio decoder ringbuf write timeout W (80835) AV_STREAM: audio decoder ringbuf write timeout W (80837) AV_STREAM: audio decoder ringbuf write timeout W (80998) AV_STREAM: audio decoder ringbuf write timeout W (81155) AV_STREAM: audio decoder ringbuf write timeout W (81189) AV_STREAM: audio decoder ringbuf write timeout W (81192) AV_STREAM: audio decoder ringbuf write timeout W (81339) AV_STREAM: audio decoder ringbuf write timeout W (81344) AV_STREAM: audio decoder ringbuf write timeout W (81345) AV_STREAM: audio decoder ringbuf write timeout W (81378) AV_STREAM: audio decoder ringbuf write timeout W (81383) AV_STREAM: audio decoder ringbuf write timeout W (81387) AV_STREAM: audio decoder ringbuf write timeout W (81535) AV_STREAM: audio decoder ringbuf write timeout W (81698) AV_STREAM: AEC reference write timeout ref 2560 W (81739) AV_STREAM: AEC reference write timeout ref 2560 W (82245) AV_STREAM: AEC reference write timeout ref 2560 I (84391) SIP: Sending keep-alive to server W (84546) AV_STREAM: AEC reference write timeout ref 2560 W (84547) AV_STREAM: audio decoder ringbuf write timeout W (84585) AV_STREAM: AEC reference write timeout ref 2560 W (84695) AV_STREAM: audio decoder ringbuf write timeout W (84697) AV_STREAM: audio decoder ringbuf write timeout W (84698) AV_STREAM: audio decoder ringbuf write timeout W (84733) AV_STREAM: audio decoder ringbuf write timeout W (84735) AV_STREAM: audio decoder ringbuf write timeout W (84736) AV_STREAM: audio decoder ringbuf write timeout I (84852) SIP: [1970-01-01/01:55:19]<<=====READ 0479 bytes== I (84857) SIP:

BYE sip:XXX:10286;transport=TCP SIP/2.0 Via: SIP/2.0/TCP XXX:5060;rport;branch=z9hG4bKPj39778cc4-05df-4a33-b316-752a7addd388;alias From: "XXX" sip:XXX;tag=08337d18-6ce1-41b2-acbe-4cae2979881c To: sip:[email protected];tag=-589368526 Call-ID: 80e647f5-f69b-4910-bf74-ce3fc42dfa22 CSeq: 14570 BYE Reason: Q.850;cause=16 Max-Forwards: 70 User-Agent: FPBX-17.0.19.25(22.2.0) Content-Length: 0

I (85090) SIP: [1970-01-01/01:55:19]<<====================== I (85206) SIP_SERVICE: ESP_RTC_EVENT_AUDIO_SESSION_END I (85227) AV_STREAM: _audio_enc task stoped I (85269) ALGORITHM_STREAM: _algo_fetch_task is stopped I (85270) AFE: exit W (85271) AUDIO_PIPELINE: There are no listener registered I (85283) AUDIO_PIPELINE: audio_pipeline_unlinked I (85285) AV_STREAM: _audio_dec task stoped I (85287) SIP_SERVICE: ESP_RTC_EVENT_HANGUP I (85288) VoIP_EXAMPLE Uart module: Send phone command I (85300) wifi:state: run -> init (0x0) I (85301) wifi:pm stop, total sleep time: 29448709 us / 80849541 us

  • Compiler version xtensa-esp-elf-gcc.exe (crosstool-NG esp-14.2.0_20241119) 14.2.0
  • Operating system: Windows
  • (Windows only) Environment type: Plain Command Prompt
  • Using an IDE?: Yes, Visual Studio Code
  • Power supply: USB|Battery

I have following test stand: ESP32 <--> FreePBX Server <--> Softphone ESP32 has custom I2S codec.

We make call from ESP32 to Softphone and vice versa, call is ok.

Expected Behavior

ESP32 speaker plays sound from Softphone microphone, Softphone speaker plays sound from ESP32 microphone.

Actual Behavior

ESP32 speaker doesn't play quality sound from Softphone microphone, Softphone speaker plays sound from ESP32 microphone. SIP channel is ok. Audio from ESP32 speakers is noisy and slurred.

Steps to Reproduce

Building standard VOIP example

Code to Reproduce This Issue

Standard VOIP example.

Hasaken avatar May 04 '25 10:05 Hasaken

Dear Espressif Team, could you please review my request?

Hasaken avatar May 11 '25 15:05 Hasaken

Can you disable AEC and have a try, seems write to reference block too long.

   av_stream_config_t av_stream_config = {
        .algo_mask = 0,// ALGORITHM_STREAM_DEFAULT_MASK,
        .acodec_samplerate = AUDIO_CODEC_SAMPLE_RATE,
        .acodec_type = AV_ACODEC_G711A,
        .vcodec_type = AV_VCODEC_NULL,
        .hal = {
            .audio_samplerate = AUDIO_HAL_SAMPLE_RATE,
            .audio_framesize = PCM_FRAME_SIZE,
        },
    };

Also you can add some log to trace the received data flow Key points: 1: audio received in static int _receive_audio(unsigned char *data, int len, void *ctx) 2: Audio decoded and started to write https://github.com/espressif/esp-adf/blob/master/examples/protocols/components/av_stream/av_stream.c#L484 3: Final audio output https://github.com/espressif/esp-adf/blob/master/examples/protocols/components/av_stream/av_stream_hal/av_stream_hal_audio.c#L180

TempoTian avatar May 13 '25 09:05 TempoTian

@TempoTian Thank you for the info. The errors below disappeared: W (74483) AFE: Ringbuffer of AFE is empty, Please use feed() to write data W (75450) AV_STREAM: AEC reference write timeout ref 2560 W (81387) AV_STREAM: audio decoder ringbuf write timeout Problem with noise instead of audio from ESP32 speakers - stays. I think there is some misconfiguration/problem with AUDIO_STREAM_WRITER, because AUDIO_STREAM_READER works good and Softphone gets audio from ESP32 microphone.

Hasaken avatar May 14 '25 04:05 Hasaken

I think you can create one echo demo, play the recorded sound directly to check whether it works or not. use int av_stream_audio_read(char *buf, int len, TickType_t wait_time, bool uac_en) to read use av_stream_audio_write to write. On most board read and write through same I2S and have same clock settings, if you are using 2 i2s need pay attention to the clock settings.

TempoTian avatar May 14 '25 04:05 TempoTian

@TempoTian Our codec is not I2S, it is PCM codec operating in mono, having 8KHz bitrate and 16 bit sample length. Could you please advice correct ADF i2s stream configuration to connect it?

Hasaken avatar May 26 '25 16:05 Hasaken

@Hasaken We have updated some code about AEC， pls try again

shootao avatar May 27 '25 12:05 shootao