Everett Kleven

@luansemensato and @ricardoschiller. Swarms runs on Python 3.10, which should explain some of the install/build issues. If you can create a new virtual environment with 3.10 and check back in...

```python import pytest import tempfile import shutil import os import daft import pyarrow as pa import lance class TestSchemaNullabilityError: """Test suite for schema nullability issues in Daft's Lance integration.""" @pytest.fixture...

@srilman excellent notes here. Concerning your example of concatenating two dataframes with fields that share the same field name and datatype with different nullability, this makes sense but may warrant...

@srilman I used the AI2D Subset from [HuggingFaceM4/the_cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron/viewer?views%5B%5D=ai2d) which stores images as png byte string. For vLLM this didn't work for me, it had to be base64.

```python import daft import base64 df_raw = daft.read_parquet('hf://datasets/HuggingFaceM4/the_cauldron/ai2d/train-00000-of-00001-2ce340398c113b79.parquet') # To Get Daft Image df = df_raw.explode(col("images")).with_column("image_png", df["images"].struct.get("bytes").image.decode()) # To Get Base64 df = df.with_column( "image_base64", df["images"].struct.get("bytes").apply( lambda x: base64.b64encode(x).decode('utf-8'), return_dtype=daft.DataType.string()...

I think that makes sense. Images aren't the only thing that require base64 encoding. Audio would need it as well for image/audio inputs to vLLM/SGLang. This helps accomplish both! Very...

Text example

This might be a google colab thing

More context for LanceDB folks: Daft has a catalog and session abstraction for referencing and managing tables. While you can currently read and write from lance tables using pylance, there...

I think since the the directory catalog is flat, it should matche closely with the recent MemoryCatalog addition: https://github.com/Eventual-Inc/Daft/pull/4445 which is backed by Rust and fully available in python. REST...