Webviewer does not show large chunks when memory limit is exceeded but blocks memory

[Famok]

Describe the bug While investigating another issue 11699 about beeing unable to increase the memory limit I noticed that pushing larger chunks of data using rr.send_columns can result in having memory allocated (according to memory panel) but not showing up. At the same time i get the message that data has been dropped. The funny thing is, that the stack of images I'm pushing here is about 1 GB when I get the "drop message" even though it says I have a memory limit of 2.3 GB.

To Reproduce Start grpc and webviewer using e.g. rerun --serve-web and push a large chunk of data in it, using rr.send_columns that causes data to be dropped.

Or use this script: (be aware that it might also crash the viewer ...)

# %%
import rerun as rr
import numpy as np

rr.init(application_id="rerun_example_serve_web_viewer", spawn=False)
# Start a gRPC server and use it as log sink.
server_uri = rr.serve_grpc(server_memory_limit="500MB")

# Connect the web viewer to the gRPC server and open it in the browser
rr.serve_web_viewer(connect_to=server_uri, open_browser=True)
img = np.random.randint(0,255,(10000,50000))
print(f"Image Size: {img.nbytes*1E-6} MB")
rr.log('test_img', entity=rr.Image(img))

# now log an image chunk:
#%%
def log_image_batch(
    entity_path: str,
    times: np.ndarray,
    image_batch: np.ndarray,
    transform: rr.Transform3D = None,
    recording: rr.RecordingStream = None,  # optional if we want to send the data to recording which is not the currently activated one
):
    """
    Log a batch of images to the specified entity path all at once.

    Parameters
    ----------
    entity_path : str
        The path where the images will be logged.
    times : np.ndarray
        An array of timestamps corresponding to each image in the batch.
    image_batch : np.ndarray
        A batch of images to be logged. The shape should be (N, H, W, C) for color images or (N, H, W) for grayscale images.
    transform : rr.Transform3D, optional
        A transformation to be applied to the images. Default is None.
    recording : rr.RecordingStream, optional
        The recording stream to which the data will be sent. Default is None.

    Returns
    -------
    None
    """

    # Log the ImageFormat and indicator once, as static.
    color_model = "RGB" if len(image_batch.shape) > 3 else "L"
    channel_datatype = "U8" if image_batch.dtype == np.uint8 else "U16"
    format_static = rr.components.ImageFormat(
        width=image_batch.shape[2],
        height=image_batch.shape[1],
        color_model=color_model,
        channel_datatype=channel_datatype,
    )

    batch = rr.components.ImageBufferBatch(
        np.frombuffer(image_batch.tobytes(), dtype=np.uint8).reshape(
            [
                image_batch.shape[0],
                image_batch[0].nbytes,
            ]
        ),
    )

    rr.log(
        entity_path,
        rr.Image.from_fields(format=format_static),
        static=True,
        recording=recording,
    )

    if transform is not None:
        rr.log(
            entity_path=entity_path,
            entity=transform,
            static=True,
            recording=recording,
        )

    # Reshape the images so `ImageBufferBatch` can tell that this is several blobs.

    rr.send_columns(
        entity_path,
        indexes=[rr.TimeColumn("time", duration=times)],
        columns=rr.Image.columns(buffer=batch),
        recording=recording if recording is not None else None,
    )

n = 2000
imgs = np.random.randint(0, 255, (n, 480, 640, 3), dtype=np.uint8)
print(f"Image batch size: {imgs.nbytes / 1e6} MB")
times = np.arange(n) * 0.1

log_image_batch("test_batch", times=times, image_batch=imgs)

Expected behavior Best result would be at least showing me all data that has been pushed, OR clear all the memory that it would have consumed.

Desktop (please complete the following information): Windows 11 Enterprise Version10.0.22631 Build 22631 System Model: HP ZBook Fury 16 G10 Mobile Workstation PC Processor: 13th Gen Intel(R) Core(TM) i7-13850HX, 2100 Mhz, 20 Core(s), 28 Logical Processor(s) Installed Physical Memory (RAM): 32,0 GB

Rerun version 0.26.2

[Wumpf]

Related discord thread https://discord.com/channels/1062300748202921994/1075873257124810852/1432998003769086063

Generally the web viewer is extremely memory constraint since it's a 32bit (wasm32) application. I'm not too surprised it having trouble with 1Gb columns but the described behavior is strange nontheless. A repro to play with would be nice :)