OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

[feat: set context window limit] being able to set the context window to < 200k

Open ryx2 opened this issue 3 months ago • 0 comments

What problem or use case are you trying to solve?

Usually, Claude does not perform well as you approach the 200k context. I would want to set it to something like ~30k when I run openhands on my code. This is ~3k lines of code and logs presuming an average of 10 tokens per line.

Describe the UX of the solution you'd like None Do you have thoughts on the technical implementation?

Use the new context truncation (https://github.com/All-Hands-AI/OpenHands/pull/4977), connected to an input flag, to allow users to set a context window smaller than the maximum allowed by the LLM

Describe alternatives you've considered

Additional context

https://github.com/All-Hands-AI/OpenHands/pull/4977

ryx2 avatar Nov 14 '24 23:11 ryx2