AFE frame AEC etc algorithm need hardware support? (AIS-1642)

[WangShuoran]

In your doc https://docs.espressif.com/projects/esp-sr/zh_CN/latest/esp32s3/audio_front_end/README.html show all of AFE algorithm.

I have ESP32-LYRAT_V4.3 want to use all of last picture algorithm. But I don't know the hardware requirements for each algorithm.

My Question is ESP32-LYRAT_V4.3 support all of the last picture algorithm?

https://dl.espressif.com/dl/schematics/ESP32-LYRAT_V4.3-20220119.pdf

The following question is a sub question of the previous question:

1. Does the AEC algorithm require ESP32S3 to have two I2S hardware connections( SPEAKER and MIC )?

[feizi]

My Question is ESP32-LYRAT_V4.3 support all of the last picture algorithm?

ESP32 does not support all algorithm. I recommend to use ESP32-S3.

1. Does the AEC algorithm require ESP32S3 to have two I2S hardware connections( SPEAKER and MIC )?

AEC need SPEAKER and MIC signal, but one I2S can get those at the same time. Please refer to esp32-s3-korvo-2.

[lihuimin01]

ESP32S3 AFE开启AEC算法后，发生不能唤醒的问题？ IDF 4.4.8 ADF是 V2.6 AEC通道的数据和MIC数据是怎么排列的，请教一下 我的初始化如下： afe_config_t afe_config = AFE_CONFIG_DEFAULT(); afe_config.memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM; afe_config.wakenet_init = true; afe_config.wakenet_model_name = wn_name; afe_config.voice_communication_init = false; afe_config.aec_init = true; afe_config.pcm_config.total_ch_num = 3; afe_config.pcm_config.mic_num = 2; afe_config.pcm_config.ref_num = 1; afe_config.pcm_config.sample_rate = 16000;

afe_config.vad_mode = VAD_MODE_4;

读取音频流并feed到AFE如下 static void wakeup_feed_Task(void *arg) { esp_afe_sr_data_t *afe_data = g_wakeup.afe_data; int audio_chunksize = g_wakeup.afe_handle->get_feed_chunksize(afe_data); int nch = g_wakeup.afe_handle->get_channel_num(afe_data);

int buff_size = audio_chunksize * sizeof(int16_t) * nch;
//int mono_buffer_size = audio_chunksize  * sizeof(int16_t);

int16_t *i2s_buff = malloc(buff_size);
//int16_t *mono_buffer = malloc(mono_buffer_size);
if (!i2s_buff) {
    DUER_LOGE("malloc error");
}
#if 0
if (!mono_buffer) {
    DUER_LOGE("malloc error");
}
#endif
DUER_LOGI("channel = %d chunksize = %d buff_size = %d\n",
     nch, audio_chunksize, buff_size);
int read_size = 0;

while (g_wakeup.task_run) {
    memset(i2s_buff, 0, buff_size);
    read_size = audio_element_input(g_wakeup.i2s_stream_reader, (char *)i2s_buff, buff_size);
    g_wakeup.afe_handle->feed(afe_data, i2s_buff);

检测唤醒任务如下: static void wakeup_detect_Task(void *arg) { esp_afe_sr_data_t *afe_data = g_wakeup.afe_data; //model_iface_data_t *nm_iface_data = g_wakeup.model_iface_data; static afe_vad_state_t local_state; static uint8_t frame_keep = 0; bool detect_flag = false; static uint32_t retry_cnt = 0;

int afe_chunksize = g_wakeup.afe_handle->get_fetch_chunksize(afe_data);
int16_t *buff = malloc(afe_chunksize * sizeof(int16_t));
printf("------------detect start------------\n");
assert(buff);
while (g_wakeup.task_run) {
    
    afe_fetch_result_t* res = g_wakeup.afe_handle->fetch(afe_data);

    if (!res || res->ret_value == ESP_FAIL) 
    {
        printf("fetch error!\n");
        if (retry_cnt++ > 10) 
        {
            duer_dcs_audio_stop_handler();
            retry_cnt = 0;
        }
        continue;
    }
    retry_cnt = 0;
    if (res->wakeup_state == WAKENET_DETECTED) 
    {
        printf("wakeword detected\n");
        if (g_wakeup.wakeup_event_cb) 
        {
            bdvs_wakeup_event_t event = {0};
            event.type = BDVS_WAKEUP_EVENT_TRIGGERED;
            g_wakeup.wakeup_event_cb(&event);
        }
    }

[sun-xiangyu]

我看你的配置

afe_config.pcm_config.mic_num = 2;
afe_config.pcm_config.ref_num = 1;

你的feed输入应该是 m1,m2,r1,m1,m2,r1,...., 其中m1 表示麦克风通道1， m2 表示麦克风通道2，r1 表示playback 回采通道