OpenAIChatCompletionChoiceMessageContentItemModel.imageUrl Not working properly

[gianlucasama]

Following code:

String encodedImage = await loadImage(imageFile);
final message = OpenAIChatCompletionChoiceMessageModel(
  content: [
    OpenAIChatCompletionChoiceMessageContentItemModel.imageUrl(
      'data:image/jpeg;base64,$encodedImage',
    ),
  ],
  role: OpenAIChatMessageRole.user,
);
final chatCompletion = await OpenAI.instance.chat.create(
  model: 'gpt-4-turbo',
  seed: 0,
  messages: [message], 
  temperature: 1,
  maxTokens: 512,
);

Throws:

RequestFailedException (RequestFailedException(message: Invalid type for 'messages[0].content[0].image_url': expected an object, but got a string instead., statusCode: 400))

Solved by changing OpenAIChatCompletionChoiceMessageContentItemModel method:

/// This method used to convert the [OpenAIChatCompletionChoiceMessageContentItemModel] to a [Map<String, dynamic>] object.
  Map<String, dynamic> toMap() {
    return {
      "type": type,
      if (text != null) "text": text,
      if (imageUrl != null) "image_url": imageUrl,
    };
  }

Into:

/// This method used to convert the [OpenAIChatCompletionChoiceMessageContentItemModel] to a [Map<String, dynamic>] object.
  Map<String, dynamic> toMap() {
    return {
      "type": type,
      if (text != null) "text": text,
      if (imageUrl != null) "image_url": {"url": imageUrl},
    };
  }

Following https://platform.openai.com/docs/guides/vision

If you need a pull request for this let me know :) (Just noticed there is already a pull request for this)

[anasfik]

I believe the issue is resolved in #166, will update the package also today, can you check?

[gianlucasama]

Yes, I can check it. I was also looking into adding the "detail" parameter so that people can manually switch between the "low" and "high" image processing schemes provided by OpenAI. From what they say the "auto" value is default: it decides automatically based on input image size. I still think we need manual control though. Being sure about input image pricing can be important.