anything-llm
anything-llm copied to clipboard
Mintplex-Labs
Support Image Uploading
The document processor should support the uploading and embedding of images like PNG, JPEG, and other static formats.
Ideally, this should describe the image and return that text for embedding instead of trying to do a multi-modal embedding which will be impossible to search textually over.
Are you thinking that it should use something like a BERT or Deepdanbooru to extract info from?
Are you thinking that it should use something like a BERT or Deepdanbooru to extract info from?
Both of these would be an issue to run locally since they require some big resources. Deepdanbooru is also specific to anime-girls image tagging and tends to give more NSFW results so honestly easiest implementation is just using something simple like OpenAIs CLIP which can run on replicate pretty easily (but will still cost money)
https://replicate.com/rmokady/clip_prefix_caption
If you are calling out to external resources, there’s lots of choices of course.
it return anythingllm File extension .jpg not supported for parsing and cannot be assumed as text file type.
@phicha20224 - that is because we dont support uploading images right now
@phicha20224 - that is because we dont support uploading images right now
