Token counting and litellm provider customization

[computer-whisperer]

Schedule monologue compression with token counting rather than character counting, preventing occasional input token count overruns that can happen when lots of non-textual output occur (creating scenarios where the same number of chars can require more tokens).

(This has a minor conflict with https://github.com/OpenDevin/OpenDevin/pull/1417, and will be updated after that pr gets merged)

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 86.66667% with 4 lines in your changes are missing coverage. Please review.

:exclamation: No coverage uploaded for pull request base (main@4e84aac). Click here to learn what that means.

Files	Patch %	Lines
opendevin/llm/llm.py	81.81%	4 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1421   +/-   ##
=======================================
  Coverage        ?   60.83%           
=======================================
  Files           ?       88           
  Lines           ?     3738           
  Branches        ?        0           
=======================================
  Hits            ?     2274           
  Misses          ?     1464           
  Partials        ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rbren]

@enyst looks like we need your 👍

[rbren]

@computer-whisperer looks like it just needs a rebase. Feel free to ping me when it's ready!

[computer-whisperer]

@rbren should be ready to go

[enyst]

This will make the behavior much better, thank you! And sorry for the delay here.