anything-llm
anything-llm copied to clipboard
[FEAT]: Add ability to rewrite parameters in the json request.
What would you like to see?
In case when "max_tokens" field is none my LLM server updates the field with too small tokens number. More precisely it is around 500 and most my coding requests got truncated. Please, add a button in the Options/LLM preferences that for each LLM can add item to an update dict with parameter and its value, like "max_tokens: 2048 or "top_k: 3". Then update json request with this dict. On the top there may be button to reset (clear update dict).
This feature will allow to fix many personal user problems by themselves and I think will be very welcomed by the community. Meanwhile, I have to edit LLM server code each time I update it, since all the changes got erased.