anthropics
Debounce Haiku background word generation by default (or maybe even turn it off) while typing
There are sometimes multiple calls a second to Haiku 3.5 to generate the mood word which I think is incredibly wasteful. I'm not being charged for these so it's not about money, but just can't justify the amount of network chatter and GPU cycles.
The mood word is cute and I'm not even saying to give an option to turn it off (though that wouldn't be a bad thing to offer either), but if you debounc the requests to every 5-10s or even just send it off when the message is finished and sent off, the response times are consistently under 2s even here in the UK so it wouldn't take away from the magic.
Edit: I'm not sure if it's related, but sometimes characters appearing after typing feels slow as if these requests were synchronous?
