dialoqbase icon indicating copy to clipboard operation
dialoqbase copied to clipboard

Published 20 hours ago • verified publisher icon n4ze3m

Token Limit

Open PyT1m opened this issue 1 year ago • 6 comments
trafficstars

Would it be possible to implement a token limiter for each chatbot. In another thread there was the suggestion to implement bot statistics as an enhancement. Maybe you could combine it with the token limiter?

PyT1m avatar Jan 08 '24 18:01 PyT1m

Hey, are you suggesting a max token length for each bot, right?

n4ze3m avatar Jan 10 '24 17:01 n4ze3m

I think that for each rebot, we should be able to adjust:

token context max token Temperature Top_P Top_K Penalty

This really changes the answers

comeback01 avatar Jan 11 '24 10:01 comeback01

The existing version supports temperature; you can change it in the settings. I will add the reset of the configuration

n4ze3m avatar Jan 11 '24 11:01 n4ze3m

For example fastbots offers plans like: 5000 Messages P/M | 15,000,000 characters/bot. So they have implemented a limit on tokens to spend per month somehow. That was my idea of a token-limitation.

PyT1m avatar Jan 11 '24 23:01 PyT1m

This must be done at provider level.

A key leaves rebot, with a limit.

The only one I've seen that offers this is https://openrouter.ai.

They even have free templates!

comeback01 avatar Jan 12 '24 10:01 comeback01

Ah okay, I see. And would it be possible to integrate openrouter.ai (openrouter.api) and the limits in dialoqbase?

PyT1m avatar Jan 12 '24 18:01 PyT1m