spring-ai
spring-ai copied to clipboard
Make allow image url for Anthropic API
trafficstars
related issue: https://github.com/spring-projects/spring-ai/issues/2804
I have improved the Anthropic API to support image URLs.
The following areas still need improvement:
- In Spring AI, the
MimeTypeis required in theMediaobject, but the Anthropic API does not require the MimeType. Therefore, we need to explicitly provide a MimeType (any value can be used).
example:
String response = ChatClient.create(this.chatModel).prompt()
.options(AnthropicChatOptions.builder().model(modelName).build())
.user(u -> u.text("Explain what do you see on this picture?").media(MimeTypeUtils.IMAGE_PNG, url))
.call()
.content();
MimeTypeUtils.IMAGE_PNG is not necessary for the Anthropic API, but it must be included to meet the requirements of Spring AI.
I thought that creating a media(URL url) method that does not require a MimeType could cause confusion with other models that mandate the use of a MimeType.