Video stream flickers on Safari and Firefox

[cdeil]

@MarcSkovMadsen - thanks for making nice tutorials on video streaming.

https://panel.holoviz.org/tutorials/intermediate/build_server_video_stream.html https://github.com/holoviz/panel/blob/main/examples/gallery/streaming_videostream.ipynb

For me on latest MacOS Safari 26.0.1 with Python 3.12 and Panel 1.8.2 they flicker on every (or most) frames. I.e. the browser doesn't keep the old frame until the new one is there. This happens at any frame rate, i.e. even at very low frame rates.

I quickly tried with latest Firefox 144.0.2 and also saw the flicker.

On Chrome it seems to work fine, so I'm using that for now for my prototyping.

Any idea what the problem is or how to fix it?

[cdeil]

I asked Gemini/ChatGPT about this and they claim that the root cause of the flicker is that the <img> src attribute is updated which triggers a "clear-decode-paint" cycle and flicker. Apparently Chrome does it differently and keep the old image until the new is in place, that's why it's not flickering.

Probably no solution exist to configure the <img> update on Safari/Firefox, and the only reliable solution would be to draw the image on <canvas> instead and not use an <img> HTML element.

In addition it might be more efficient to encode to JPEG and decode in the browser to have less data on the wire. Although probably both approaches (raw bytes or encoded) will run into performance issues at high resolution and FPS.

I guess such approaches could be new alternative image components, the existing panel.pane.Image <-> <img> should stay as-is and this isn't really a (solvable) bug?

[philippjfr]

Thanks for the investigation, I think your instinct that there isn't a good way forward with Image is correct. It's not too difficult to build a custom Canvas based component (GPT-5 spit this working version out on the first attempt):

import param
import panel as pn
from panel.custom import JSComponent

pn.extension()

class ImageStream(JSComponent):
    """Stream image bytes from Python -> JS -> <canvas>."""
    # Raw bytes of the image (PNG/JPEG/WebP...). Set this from Python.
    bytes = param.Bytes(default=b"")

    # MIME used when constructing the Blob on the JS side
    mime = param.ObjectSelector(default="image/png",
                                objects=["image/png", "image/jpeg", "image/webp"])

    # Optional CSS border toggle (just to see the canvas bounds)
    show_border = param.Boolean(default=True, doc="Show a 1px border around the canvas.")

    _esm = """
export function render({model, el}) {
  // --- Elements
  const canvas = document.createElement('canvas');
  const ctx = canvas.getContext('2d', { willReadFrequently: false });
  const wrapper = document.createElement('div');
  wrapper.style.display = 'inline-block';
  wrapper.appendChild(canvas);
  el.appendChild(wrapper);

  // --- Sizing / border
  function syncSizeAndStyle() {
    canvas.width  = model.width ?? 300;
    canvas.height = model.height ?? 150;
    canvas.style.display = 'block';
    canvas.style.border = model.show_border ? '1px solid #ddd' : 'none';
  }

  // Convert whatever we get into a Uint8Array
  function toUint8Array(input) {
    if (!input) return null;

    if (input instanceof ArrayBuffer) return new Uint8Array(input);

    if (ArrayBuffer.isView(input)) {
      return new Uint8Array(input.buffer, input.byteOffset, input.byteLength);
    }

    if (Array.isArray(input)) {
      return new Uint8Array(input);
    }

    if (typeof input === 'string') {
      // Assume base64 (Panel will already use a Bytes transport, but be permissive)
      try {
        const bin = atob(input);
        const out = new Uint8Array(bin.length);
        for (let i = 0; i < bin.length; i++) out[i] = bin.charCodeAt(i);
        return out;
      } catch (e) {
        console.warn('ImageStream: could not base64-decode string bytes', e);
        return null;
      }
    }

    console.warn('ImageStream: unsupported bytes payload', input);
    return null;
  }

  // Draw preserving aspect ratio and centered
  async function drawFromBytes() {
    const u8 = toUint8Array(model.bytes);
    const cw = canvas.width, ch = canvas.height;

    // Clear canvas first
    ctx.clearRect(0, 0, cw, ch);

    if (!u8 || u8.length === 0) return;

    try {
      const mime = model.mime || 'image/png';
      const blob = new Blob([u8], { type: mime });
      const bitmap = await createImageBitmap(blob);

      const scale = Math.min(cw / bitmap.width, ch / bitmap.height);
      const dw = Math.max(1, Math.round(bitmap.width * scale));
      const dh = Math.max(1, Math.round(bitmap.height * scale));
      const dx = Math.floor((cw - dw) / 2);
      const dy = Math.floor((ch - dh) / 2);

      ctx.imageSmoothingEnabled = true;
      ctx.imageSmoothingQuality = 'high';
      ctx.drawImage(bitmap, dx, dy, dw, dh);
    } catch (err) {
      console.error('ImageStream: failed to draw image bytes', err);
    }
  }

  // Initial render
  syncSizeAndStyle();
  drawFromBytes();

  // Reactivity
  model.on(['width', 'height', 'show_border'], () => {
    syncSizeAndStyle();
    // re-draw current frame to updated size
    drawFromBytes();
  });

  model.on(['bytes', 'mime'], () => {
    drawFromBytes();
  });
}
"""