[bug] "Other field types of parts are not supported yet" error when attaching images in GoogleGenAIInstrumentor

[justinas-kazanavicius]

Describe the bug The openinference-instrumentation-google-genai package throws an error "Other field types of parts are not supported yet" when processing image data attached to Google GenAI requests. This error occurs specifically in the _get_attributes_from_part function at line 367 of the request attributes extractor, suggesting that the instrumentation library doesn't properly handle types.Part objects created from base64 image data.

To Reproduce

import base64
import asyncio
from google import genai
from google.genai import types
from openinference.instrumentation.google_genai import GoogleGenAIInstrumentor

# Enable instrumentation (this is what causes the error)
GoogleGenAIInstrumentor().instrument()

async def reproduce_error():
    client = genai.Client(api_key="your-api-key")
    
    # Create a simple base64 image (1x1 pixel PNG)
    base64_img = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
    
    # Extract MIME type and decode base64 data
    mime_type = base64_img.split(";")[0].replace("data:", "")
    img_data = base64.b64decode(base64_img.split(",")[1])
    
    # Create Part object from bytes (this is what triggers the error)
    image_part = types.Part.from_bytes(
        data=img_data,
        mime_type=mime_type,
    )
    
    parts = ["Describe this image", image_part]
    
    # This call will trigger the instrumentation error
    response = await client.aio.models.generate_content(
        model="gemini-2.5-flash",
        contents=parts,
        config=types.GenerateContentConfig(temperature=0.0),
    )
    return response

# Run the reproduction
asyncio.run(reproduce_error())

Expected behavior The request should complete successfully without instrumentation errors. The OpenInference instrumentation should properly handle types.Part objects created from both from_bytes() and from_uri() methods without throwing unsupported field type errors.

Error Details

{
  "dt": "2025-09-11T11:52:33.622395+00:00",
  "level": "error", 
  "severity": 4,
  "message": "Other field types of parts are not supported yet",
  "context": {
    "runtime": {
      "function": "_get_attributes_from_part",
      "file": "../lib/python3.13/site-packages/openinference/instrumentation/google_genai/_request_attributes_extractor.py",
      "line": 367,
      "thread_id": 139450478468032,
      "thread_name": "MainThread",
      "logger_name": "openinference.instrumentation.google_genai._request_attributes_extractor"
    }
  }
}

Environment:

• OS: macOS (Darwin 24.6.0) (but also in prod with debian)
• Python: 3.13
• google-genai: 1.36.0
• openinference-instrumentation-google-genai: 0.1.5

Root Cause Analysis The error suggests that the OpenInference instrumentation library's _get_attributes_from_part function doesn't handle all possible types.Part variations. Specifically, it appears to not support Part objects created via types.Part.from_bytes() method, which is commonly used for base64-encoded image data.

Related Issue Previously encountered similar error: "Unexpected input contents type: <class 'PIL.JpegImagePlugin.JpegImageFile'>" when using PIL Image objects directly instead of types.Part.from_bytes(). This was resolved by replacing PIL usage with types.Part.from_bytes() in commit c476ea9d, but the instrumentation library still doesn't properly handle the resulting Part objects.

Additional context

• Error appears as: ERROR:openinference.instrumentation.google_genai._request_attributes_extractor:Other field types of parts are not supported yet
• The error is non-blocking but creates noise in logs and monitoring
• Affects all image processing workflows using base64 image data
• API calls complete successfully despite the instrumentation error

[dosubot[bot]]

Hi @justinas-kazanavicius! I'm Dosu and I’m helping the openinference team.

The error occurs because the instrumentation's _get_attributes_from_part function only supports extracting 'text' fields from types.Part objects. When it encounters a part created from bytes (like an image via from_bytes()), it logs "Other field types of parts are not supported yet" and does not process the image part further. This is a current limitation—support for image/bytes parts (including those from from_bytes() or from_uri()) is not implemented in the instrumentation logic, which is why you see this error in your logs. There are no configuration options or environment variables available to suppress this error log, and no open pull requests or documentation indicate that support for these part types is planned yet. If the log noise is problematic, you might consider filtering or suppressing this specific log message at the logger configuration level in your application until official support is added.
Reference: code analysis

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[justinas-kazanavicius]

I'm using this workaround for now:

# Monkey-patch to fix OpenInference instrumentation image logging
from collections.abc import Iterator
from openinference.instrumentation.google_genai import _request_attributes_extractor
from openinference.instrumentation.google_genai._request_attributes_extractor import MessageAttributes

def patched_get_attributes_from_part(self, part: Any) -> Iterator[tuple[str, Any]]:
    """Patched version that handles image parts properly."""
    # Check for text (original behavior)
    if hasattr(part, 'text') and part.text:
        yield (MessageAttributes.MESSAGE_CONTENT, part.text)
    # Handle inline_data (base64 images)
    elif hasattr(part, 'inline_data') and part.inline_data:
        mime_type = part.inline_data.mime_type if part.inline_data.mime_type else "unknown"
        data_size = len(part.inline_data.data) if part.inline_data.data else 0
        yield (MessageAttributes.MESSAGE_CONTENT, f"[Image: {mime_type}, {data_size} bytes]")
    # Handle file_data (URI references)
    elif hasattr(part, 'file_data') and part.file_data:
        file_uri = part.file_data.file_uri if part.file_data.file_uri else "unknown"
        mime_type = part.file_data.mime_type if part.file_data.mime_type else "unknown"
        yield (MessageAttributes.MESSAGE_CONTENT, f"[Image: {mime_type} from {file_uri}]")

# Apply the monkey patch
_request_attributes_extractor._RequestAttributesExtractor._get_attributes_from_part = patched_get_attributes_from_part

# Now instrument normally
GoogleGenAIInstrumentor().instrument()

[delatt]

Just to add that this same issue occurs when FunctionCalls are returned from the model as there is no text attribute on the part, would it make sense to simply ignore any parts with a None text attribute?

[guillaumefrd]

I have the same issue when I add a PDF to the contents of the API request using:

types.Part.from_bytes(
    data=filepath.read_bytes(),
    mime_type='application/pdf',
)

[nate-mar]

Thanks for the report here! we'll try to get this on the board for next week.