anything-llm
anything-llm copied to clipboard
Mintplex-Labs
Remove `maxLength` in chat input box to allow larger prompts
Can't send larger messages because of this artificial limit.
ChatGPT also doesn't have any upper limit to what we can send as message.
The issue with doing this will be that when we have to do manual similarity searches across a vector space we have to pre-chunk the query if its over a certain length to vectorize. Turning this off like this will result in someone sending a message that will fail because the query will be too long to do a similarity search without pre-chunking for when we are not using langchain - which is just lance
Okay, now I understand. But to really make this an alternative to ChatGPT, we need to increase this limit.
I will think about this and take out some time over the weekend to improve this.
Though I'd need your help to point me in the right direction.
If you can share some action points (a list) that'd be great. That way someone in the community reading this can also chip in and help.
Split the query into multiple parts to make prompts and then generate text summarizations. Use those to piece together whatever the original message says and feed that in to generate results.
For better results you would want to generate many prompts from the original message with gradient slicing, so that the text gets split in random places and there's a chance to recover context lost by cutting sentences in the wrong place.
to work around this (because 240 was pretty unusable) I changed it to 500 on my local instance. 500 is generally ok.
500 seems to work ok here too. @preeteshjain I get the problem and I've increased the value by myself before actually looking here and see why it was so small. I'm not in the team, but I would like to say that "make this an alternative to ChatGPT" is not the goal here. This project (for now at least) has the big plus on making easy to make your own personal knowledge dataset and "talk to your documents". Until now this is the best project I've seen around that actually works and only requires one paying API: the openai's.
