langchain4j icon indicating copy to clipboard operation
langchain4j copied to clipboard

Support PDF File Input for Openai

Open jrsperry opened this issue 5 months ago • 1 comments

OpenAI models support PDF file inputs, but the openai integration currently does not support PDF contents. We should support this added capability. Here's an example http request from their playground following attaching a pdf.

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "get me the 3 most expensive products"
        },
        {
          "type": "file",
          "file": {
            "file_data": "data:application/pdf;base64,JVBERi0x.....",
            "filename": "Sears Craftsman Power and Hand Tools 1994-95_0001.pdf"
          }
        }
      ]
    }
  ],
  "response_format": {
    "type": "text"
  },
  "temperature": 1,
  "max_completion_tokens": 2048,
  "top_p": 1,
  "frequency_penalty": 0,
  "presence_penalty": 0
}'

jrsperry avatar Jun 06 '25 17:06 jrsperry

Can open AI not customize parameters? What if the parameters of the large model are encapsulated and not supported? Cannot extract interfaces or rewrite interface classes to implement custom parameters

smileyboy2019 avatar Jun 07 '25 10:06 smileyboy2019

@smileyboy2019 check out the PR to see more details.

jrsperry avatar Jun 09 '25 15:06 jrsperry

implemented in https://github.com/langchain4j/langchain4j/pull/3177

dliubarskyi avatar Jun 10 '25 11:06 dliubarskyi