IOS 平台 VideoDecoder/MediaAudioPlayer 不能正常播放音视频

[enginespot]

Bug 描述

在官方 Demo 中（例如 mesh4DPlayer.js @roamzhen @xxxBW ）提供了 createVideoDecoder 的示例代码，但未见关于 createMediaAudioPlayer 的示例。基于文档 [wx.createMediaAudioPlayer](https://developers.weixin.qq.com/miniprogram/dev/api/media/audio/wx.createMediaAudioPlayer.html) 的介绍，我尝试将视频与音频同时播放。测试结果显示：

• Android 平台：视频与音频均可正常播放
• iOS 平台：音视频不能正常播放

---

重现步骤

1. 在页面中添加 <canvas> 标签用于视频显示：

   <canvas id="videoCanvas" type="2d" style="width:300px;height:200px;"></canvas>
   

2. 在 onReady 生命周期中，通过 wx.createSelectorQuery() 获取 Canvas 节点，并初始化视频解码器和音频播放器：

   'use strict';
   Page({
     data: {},
     onLoad() {},
     onReady() {
       const query = wx.createSelectorQuery();
       query.select("#videoCanvas")
         .node()
         .exec((res) => {
           const canvas = res[0].node;
           const ctx = canvas.getContext("2d");
           const decoder = wx.createVideoDecoder();
           const audio = wx.createMediaAudioPlayer();
           let started = false;
           let imageWidth = 1280;
           let imageHeight = 720;
   
           decoder.on('start', () => {
             started = true;
           });
           decoder.on('stop', () => {
             console.info('Video decoding stopped'); 
             decoder.seek(0);
           });
           decoder.on('seek', () => {
             console.log('seek');
           });
           decoder.on('bufferchange', (info) => {
             console.log('bufferchange', info);
           });
           decoder.on('ended', () => {
             console.info('Video ended, restarting...');
           });
   
           const loop = () => {
             if (started) {
               const frameData = decoder.getFrameData();
               if (frameData && frameData.data) {
                 const imageData = ctx.createImageData(imageWidth, imageHeight);
                 imageData.data.set(new Uint8ClampedArray(frameData.data));
                 ctx.putImageData(imageData, 0, 0);
               }
             }
             canvas.requestAnimationFrame(loop);
           };
   
           (async () => {
             try {
               const decoderResult = await decoder.start({
                 source: 'https://baikebcs.bdimg.com/baike-other/big-buck-bunny.mp4',
                 mode: 0
               });
               imageWidth = decoderResult.width;
               imageHeight = decoderResult.height;
               console.log('Video decoder started:', decoderResult);
   
               const audioResult = await audio.start();
               console.log('Audio player started:', audioResult);
               await audio.addAudioSource(decoder);
             } catch (error) {
               console.error('音视频初始化出错:', error);
             }
           })();
           loop();
         });
     },
     onUnload() {}
   });
   

3. 分别在 Android 与 iOS 设备上运行该示例。

---

预期结果

视频与音频均能正常播放，无论是在 Android 还是 iOS 平台。

---

实际结果

• Android：视频与音频均可正常播放。
• iOS：音频正常播放，但视频无法显示。

---

其他提示

• 微信小游戏的音视频播放示例（参考链接：[微信小游戏音视频示例] 能够正常工作，其代码示例如下：

  import createRenderer from './render';
  
  const canvas = wx.createCanvas();
  const ctx = canvas.getContext('2d');
  const systemInfo = wx.getSystemInfoSync();
  const { windowWidth, windowHeight, pixelRatio } = systemInfo;
  canvas.width = windowWidth * pixelRatio;
  canvas.height = windowHeight * pixelRatio;
  const offcanvas = wx.createCanvas();
  
  const decoder = wx.createVideoDecoder();
  const audio = wx.createMediaAudioPlayer();
  let render;
  
  const loop = () => {
    const frameData = decoder.getFrameData();
    console.log('getFrameData', !!frameData);
    if (frameData) {
      ctx.clearRect(0, 0, canvas.width, canvas.height);
      const { width, height, data } = frameData;
      // 通过渲染函数处理视频数据
      render(new Uint8Array(data), width, height);
      ctx.drawImage(offcanvas, 0, 0, canvas.width, height / width * canvas.width);
    }
    requestAnimationFrame(loop);
  };
  
  decoder.start({ source: 'https://baikebcs.bdimg.com/baike-other/big-buck-bunny.mp4', mode: 0 }).then(data => {
    wx.setPreferredFramesPerSecond(data.fps);
    render = createRenderer(offcanvas, data.width, data.height);
    audio.start().then((res) => {
      console.log('audio start', res);
      audio.addAudioSource(decoder).then((res2) => {
        console.log('audio add', res2);
        loop();
      });
    });
  });
  

• 该示例在 iOS 平台上视频与音频均能正常播放

---

期待协助排查具体原因并提供修复建议。谢谢！

---