spring-ai icon indicating copy to clipboard operation
spring-ai copied to clipboard

Make allow image url for Anthropic API

Open dev-jonghoonpark opened this issue 7 months ago • 0 comments
trafficstars

related issue: https://github.com/spring-projects/spring-ai/issues/2804

I have improved the Anthropic API to support image URLs.


The following areas still need improvement:

  • In Spring AI, the MimeType is required in the Media object, but the Anthropic API does not require the MimeType. Therefore, we need to explicitly provide a MimeType (any value can be used).

example:

String response = ChatClient.create(this.chatModel).prompt()
        .options(AnthropicChatOptions.builder().model(modelName).build())
        .user(u -> u.text("Explain what do you see on this picture?").media(MimeTypeUtils.IMAGE_PNG, url))
        .call()
        .content();

MimeTypeUtils.IMAGE_PNG is not necessary for the Anthropic API, but it must be included to meet the requirements of Spring AI.

I thought that creating a media(URL url) method that does not require a MimeType could cause confusion with other models that mandate the use of a MimeType.

dev-jonghoonpark avatar Apr 21 '25 08:04 dev-jonghoonpark