add example of how to upload image for GPT4 vision in README.md

[jkyngan]

First ,thanks for this amazing project. As GPT-4 vision chat completion endpoint was introduced in v0.7.8, an update of example in README.md would be great.

[ThibautPV]

Here is an example :

$result = OpenAI::chat()->create([
        'model' => 'gpt-4-vision-preview',
        'messages' => [
            [
                'role' => 'user',
                'content' => [
                    ['type' => 'text', 'text' => "What’s in this image?"],
                    ['type' => 'image_url', 'image_url' => "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"],
                ],
            ]
        ],
        'max_tokens' => 900,
    ]);

return $result->choices[0]->message->content;

[MohammadaliMirhamed]

is it possibe to upload video for it ?

[vesper8]

@MohammadaliMirhamed Vision does not support video, but what people are doing is splitting up their videos into still frames and uploading that.

[aconital]

Is there a way to upload the images? A lot of times it cannot read an image from a URL and image has to be uploaded.

[ThibautPV]

If I understood correctly, you need to upload an image to your server.

Then, you can analyze it with GPT Vision

[MohammadaliMirhamed]

I have sent https://m.media-amazon.com/images/I/81nUFx9sXoL.AC_UF894,1000_QL80.jpg to GPT-4-VISION-PREVIEW and asked for what time is it . and it responded The time on the clock is 10:10.

[aconital]

It seems we can directly pass the base64 data to the api instead of url according to the doc here https://platform.openai.com/docs/guides/vision/quick-start. Here is an example:

import base64
import requests

# OpenAI API Key
api_key = "YOUR_OPENAI_API_KEY"

# Function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')

# Path to your image
image_path = "path_to_your_image.jpg"

# Getting the base64 string
base64_image = encode_image(image_path)

headers = {
  "Content-Type": "application/json",
  "Authorization": f"Bearer {api_key}"
}

payload = {
  "model": "gpt-4-vision-preview",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What’s in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": f"data:image/jpeg;base64,{base64_image}"
          }
        }
      ]
    }
  ],
  "max_tokens": 300
}

[GilbertrdzDev]

Here I share the PHP version of @aconital:

function encodeImage($imagePath): string {
    $imageContent = file_get_contents($imagePath);
    return base64_encode($imageContent);
}

$imagePath = '/path/to/image.jpg';
$base64Image = encodeImage($imagePath);

$payload = [
    'model'      => 'gpt-4-vision-preview',
    'messages'   => [
        [
            'role'    => 'user',
            'content' => [
                [
                    'type' => 'text',
                    'text' => "What’s in this image?"
                ],
                [
                    'type' => 'image_url',
                    'image_url' => "data:image/jpeg;base64,$base64Image"
                ],
            ],
        ]
    ],
    'max_tokens' => 200,
];

$result  = OpenAI::chat()->create($payload);

echo "<figure style='font-family:sans-serif;width: 500px'>
         <img style='width: 100%' src='$imagePath' alt=''>
         <figcaption>{$result->choices[0]->message->content}</figcaption>
      </figure>";

Result:

[LintonAchmad]

updated payload for php now expecting an object for the image_url - gpt-4o:

$payload = [
            'model'      => 'gpt-4o',
            'messages'   => [
                [
                    'role'    => 'user',
                    'content' => [
                        [
                            'type' => 'text',
                            'text' => "What’s in this image?"
                        ],
                        [
                            'type' => 'image_url',
                            'image_url' => [
                                'url' => "data:image/png;base64,$base64Image"
                            ]
                        ],
                    ],
                ]
            ],
            'max_tokens' => 200,
 ];

[arhen]

updated payload for php now expecting an object for the image_url - gpt-4o:

$payload = [
            'model'      => 'gpt-4o',
            'messages'   => [
                [
                    'role'    => 'user',
                    'content' => [
                        [
                            'type' => 'text',
                            'text' => "What’s in this image?"
                        ],
                        [
                            'type' => 'image_url',
                            'image_url' => [
                                'url' => "data:image/png;base64,$base64Image"
                            ]
                        ],
                    ],
                ]
            ],
            'max_tokens' => 200,
 ];

Yeah, the error is like this:

{"error":"Invalid type for 'messages[0].content[0].image_url': expected an object, but got a string instead."}