ExoPlayer.setVideoEffects causes freezes etc

[steelbytes]

media3 1.2.1 Pixel7Pro/Android14

ExoPlayer.setVideoEffects causes freezes in DefaultVideoFrameProcessor.flush() or MediaCodec$CodecException("client does not own the buffer") when a video effect is a bit slow in TextureOverlay.getTextureId (depends how slow and if the user clicks skip back/forwards or waits for it to die).

A video effect can legitimately be a bit slow if the effect changes every frame.

ExoPlayer should instead drop/skip frames. Freezing or dying is not a result I can give to my users.

If my effect is fast then it doesn't have problems. Also using Transformer it is completely fine..

Simple repro: take the demo app and add a TextOverlay video effect to the ExoPlayer that I've made slow with some pointless loops to simulate some processing every frame (in my real code I render to a Bitmap and transfer that into a GL texture). Testing with the 'One hour frame counter (mp4)' selection that is built into the demo app.

add this to PlayerActivity.initializePlayer

        TextOverlay overlay = new TextOverlay() {
        private final SpannableString text = new SpannableString("test ");// + id);
        private final byte[] x = new byte[1000];
        @Override public SpannableString getText(long presentationTimeUs) { return text; }
        @Override public int getTextureId(long presentationTimeUs) throws VideoFrameProcessingException {
          for (int k = 0; k < 1000; ++k)
            for (int j = 0; j < 1000; ++j)
              for (int i = 0; i < x.length; ++i)
                x[i] = (byte)(x[i] + 1);
          return super.getTextureId(presentationTimeUs);
        }
      };
      List<Effect> effects = new ArrayList<>();
      effects.add(new OverlayEffect(new ImmutableList.Builder<TextureOverlay>().add(overlay).build()));
      player.setVideoEffects(effects);
      };
      List<Effect> effects = new ArrayList<>();
      effects.add(new OverlayEffect(new ImmutableList.Builder<TextureOverlay>().add(overlay).build()));
      player.setVideoEffects(effects);