David Katz
David Katz
I'll take a crack at the merge conflicts.
Fixed the conflicts/ci + pushed a small change to include these new tokens in the message stream. I think generally in good shape, if anyone else wants to take another...
Did some more testing. @mikehw would you mind taking another pass at the provider specific code. I'm seeing very low token counts for the current session input tokens (e.g.
@DOsinga I'm thinking of potentially merging everything except the provider specific code here to get the cached_token mechanism in, and then we can more incrementally tackle those.
Closing for now as some time has passed but I'm going to pick this feature back up.
https://github.com/block/goose/pull/5694 lays the groundwork for a model database, which we could have a field to check which endpoint to use.
> I can't wrk out how to do streaming, and don't want to maintain this, looking for a volunteer to pick it up @cbruyndoncx @katzdave No worries I can pick...
https://github.com/block/goose/pull/5837 have a streaming version generally working now. Working on cleaning it up.
> can I close this one @katzdave in favor of yours? Yep. Pushing on that some more today.
FYI, noticed one more issue: [Joke request.json](https://github.com/user-attachments/files/23215449/Joke.request.json) Since I changed manual compaction to actually send a message, we're now picking up that message (as the most recent user message) and...