Can not add an embedding with the .docx format.

[Vardan9898]

ErrorException: Undefined array key "data" in /home/.../vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/OpenAI/AbstractOpenAIEmbeddingGenerator.php:117

sample1.docx

Code:

$reader = new FileDataReader($filePath, PlaceEntity::class); $documents = $reader->getDocuments(); $splittedDocuments = DocumentSplitter::splitDocuments($documents, 1024, "\n"); $formattedDocuments = EmbeddingFormatter::formatEmbeddings($splittedDocuments); $embededDocuments = $embeddingGenerator->embedDocuments($formattedDocuments);

I use OpenAI3SmallEmbeddingGenerator()

[Vardan9898]

JsonException: Malformed UTF-8 characters, possibly incorrectly encoded in /home/.../vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/OpenAI/AbstractOpenAIEmbeddingGenerator.php:111

Another error when uploading pdf file

[f-lombardo]

JsonException: Malformed UTF-8 characters, possibly incorrectly encoded in /home/.../vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/OpenAI/AbstractOpenAIEmbeddingGenerator.php:111

Another error when uploading pdf file

I think that this is an error related to the smalot/pdfparser library used for parsing PDF. One option would be to open an issue there. Anyway, could you please provide a sample PDF file that generates this issue? Thank you.

[f-lombardo]

ErrorException: Undefined array key "data" in /home/.../vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/OpenAI/AbstractOpenAIEmbeddingGenerator.php:117

I created a PR to try to fix this issue: https://github.com/theodo-group/LLPhant/pull/200

Please check if it works for you

[Vardan9898]

Seems it's fixed for docx, but could you please check with attached pdf file? I get an The-No-Funnel-Strategy.pdf error:

JsonException: Malformed UTF-8 characters, possibly incorrectly encoded in /home/.../vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/OpenAI/AbstractOpenAIEmbeddingGenerator.php:111

[f-lombardo]

Seems it's fixed for docx, but could you please check with attached pdf file?

I pushed a new commit to the previous PR. Can you please check if it works for you?

[Vardan9898]

It works. Thank you.

[MaximeThoonsen]

Thanks @Vardan9898 for the issue and @f-lombardo for the PR!