[BUG] ipc timed out for 0x13020003|0x0: GLB_SET_PIPELINE_STATE when multiple pause resume on TGLU_RVP_NOCODEC_CAVSIPC4

[miRoox]

Describe the bug CI observed ipc timed out for 0x13020003|0x0: GLB_SET_PIPELINE_STATE when multiple pause resume on TGLU_RVP_NOCODEC_CAVSIPC4

CI report ID: 15143

To Reproduce

TPLG=/lib/firmware/intel/avs-tplg/cavs-tgl-nocodec.tplg MODEL=TGLU_RVP_NOCODEC_CAVSIPC4 ~/sof-test/test-case/multiple-pause-resume.sh -r 50

Reproduction Rate Only twice.

Environment

1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
   • Kernel: https://github.com/thesofproject/linux/commit/91a177700173
   • FIrmware: IPC4 tester
2. Name of the topology file
   • Topology: cavs-tgl-nocodec.tplg
3. Name of the platform(s) on which the bug is observed.
   • Platform: TGLU_RVP_NOCODEC

Screenshots or console output

dmesg

[ 2698.777318] kernel: snd_sof:sof_ipc4_set_pipeline_state: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc4 set pipeline 2 state 3
[ 2698.777328] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx      : 0x13020003|0x0: GLB_SET_PIPELINE_STATE
[ 2698.817127] kernel: snd_sof:sof_ipc4_set_pipeline_state: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc4 set pipeline 8 state 3
[ 2699.284900] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc timed out for 0x13020003|0x0
[ 2699.284908] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: preventing DSP entering D3 state to preserve context
[ 2699.284911] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump start ]------------
[ 2699.284936] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: hda irq intsts 0x00000000 intlctl 0xc0000081 rirb 00
[ 2699.284940] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: dsp irq ppsts 0x00000000 adspis 0x00000000
[ 2699.284966] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: Host IPC initiator: 0x93020003|0x0|0x0, target: 0x0|0x0|0x0, ctl: 0x3
[ 2699.284971] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump end ]------------
[ 2699.284973] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[ 2699.284975] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: IPC timeout
[ 2699.284977] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: fw_state: SOF_FW_BOOT_COMPLETE (6)
[ 2699.284998] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: 0x20000005: module: BASE_FW, state: FW_ENTERED, running
[ 2699.285019] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: status code: 0xbc0 (unknown)
[ 2699.285040] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: extended rom status:  0x20000005 0xbc0 0x8120 0x0 0x10001 0x0 0x25101c2 0x0
[ 2699.285042] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[ 2699.285055] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ASoC: error at soc_dai_trigger on SSP0 Pin: -110
[ 2699.285065] kernel:  Port0: ASoC: dpcm_be_dai_trigger() failed at NoCodec-0 (-110)
[ 2699.285070] kernel:  Port0: ASoC: trigger FE cmd: 3 failed: -110
[ 2699.285074] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx      : 0x13080003|0x0: GLB_SET_PIPELINE_STATE
[ 2699.285079] kernel: snd_sof:sof_pcm_trigger: sof-audio-pci-intel-tgl 0000:00:1f.3: pcm: trigger stream 0 dir 0 cmd 4
[ 2699.285093] kernel: snd_sof_intel_hda_common:hda_dsp_stream_trigger: sof-audio-pci-intel-tgl 0000:00:1f.3: FW Poll Status: reg[0x160]=0x14001e successful

full_dmesg.txt

[ujfalusi]

A combination of https://github.com/thesofproject/linux/pull/3843 and https://github.com/thesofproject/sof/pull/6275 should fix most IPC timeouts with IPC4.

[ujfalusi]

The firmware likely crashed: Host IPC initiator: 0x93020003|0x0|0x0, target: 0x0|0x0|0x0, ctl: 0x3

The kernel can not do much about this.

[keqiaozhang]

Observed this issue again in inner test 15477.

[ujfalusi]

@keqiaozhang, that's OK, but the kernel still can not do much about. The firmware likely crashed as it is not even acked the message. A DSP power off / on cycle might be able to fix this. What leads to a crash might be caused by the kernel doing something unexpected, but never the less, the firmware must not crash int hat case.

Can this be moved to firmware?

[keqiaozhang]

Can this be moved to firmware?

We observed this issue with IPC4 tester, not SOF firmware.

[keqiaozhang]

This is a IPC4 tester bug, not sof. Closing this bug.