[feat] Support for image inputs in prompts

[whyrusleeping]

Description Allow passing images as a parameter to the model for models that support multimodal inputs

Why is this needed I'd like to use guardrails for structured image processing

Implementation details Seems like its a small amount of wiring to pass things through but i don't really know.

End result Ideally something like:

    raw, validated_output, *rest = guard(
        litellm.completion,
        model="claude-3-haiku-20240307",
        prompt_params={
            "image": img, # PIL image as input
            "annotation": ann,
        },
        max_tokens=1024,
    )

[zsimjee]

Hi! This is a great idea, I'll look through litellm docs to see if this is supported currently.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 14 days with no activity.