Cost-Based Alert System for API Usage

[Yash-1511]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

1. Is this request related to a challenge you're experiencing?

While not a current challenge, I believe a cost-based alert system for API usage would be a valuable addition to Dify. This feature would allow for proactive management and cost control of API usage.

2. Describe the feature you'd like to see

I propose the implementation of a simple cost-based alert system. This system would monitor API usage and trigger alerts based on cost thresholds. The specifics of how these alerts are triggered and delivered can be determined during the development process.

3. How will this feature improve your workflow or experience?

This feature would significantly enhance cost management for Dify users. By providing alerts based on API usage costs, users can better manage their resources and avoid unexpected expenses.

4. Additional context or comments

At this time, I do not have additional suggestions for how this feature should be implemented. However, I am open to discussions and brainstorming sessions to refine this feature further.

5. Can you help us with this feature?

• [X] I am interested in contributing to this feature.

[dosubot[bot]]

🤖

That's a fantastic idea! Implementing a cost-based alert system for API usage would indeed be a valuable addition to Dify. Your proposal to monitor API usage and trigger alerts based on cost thresholds shows great potential for enhancing cost management for Dify users. Your willingness to contribute to this feature is highly appreciated. Your input and efforts will surely help in making Dify even more robust and user-friendly. Thank you for your willingness to contribute!

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[takatost]

duplicate #2088

[gtl-madhav]

@takatost -

you marked duplicated but the question or the feature is different here and not just a token count.

"monitor the amount of tokens used since the api call for create message does not return the token used (input and output)"

-- here we have multiple LLM and try to understand a use case, where we are using 5 different LLM's for 5 different features under 1 use case, and now I want to track all LLM's COST and soft Limit and Hard limit of that at one place in UI.

which make sense, since customer will not go on each account and check there Limit reached or not and that is what @Yash-1511 was mentioning in his request, I am guessing but he can say more about it?

[Yash-1511]

Thank you, @gtl-madhav, for the clarification on this feature.

In your explanation, you correctly outlined the method for tracking all costs in one place. I'd like to expand on this with a suggestion for an advanced feature - the ability to set usage limits, similar to what OpenAI implements for their API. This feature would enable users to establish a cap on their API usage, ensuring they stay within their budget. Additionally, it would be beneficial to have a threshold alert system. When usage approaches this predetermined threshold, users would receive notifications, perhaps via email or another preferred method. This would allow for proactive management of resources, ensuring that users are alerted before reaching their monthly budget limit. Implementing these features would significantly enhance user control and budget management.

[takatost]

This is a valuable feature that we've previously discussed, including logs, stats, alerts, cache and retries of model calls. Unfortunately, it's not currently high on our priority list. You can submit your feature requests at https://feedback.dify.ai. Our PM will evaluate this. Thanks a ton! 😊

[Yash-1511]

Thankyou @takatost for your response and considering my feature request! I understand that it's not high on the priority list at the moment, but I appreciate your openness to community suggestions.

In the meantime, I've begun exploring possible approaches for integrating this feature, including its structure and implementation plan. I look forward to potentially discussing these ideas further in the future.

Also, thanks to @gtl-madhav for their valuable insights. I'm eager to see more of your ideas on this and how we might work together to bring this feature to dify. Thanks again to everyone involved for your attention to this matter!

[takatost]

Thankyou @takatost for your response and considering my feature request! I understand that it's not high on the priority list at the moment, but I appreciate your openness to community suggestions.

In the meantime, I've begun exploring possible approaches for integrating this feature, including its structure and implementation plan. I look forward to potentially discussing these ideas further in the future.

Also, thanks to @gtl-madhav for their valuable insights. I'm eager to see more of your ideas on this and how we might work together to bring this feature to dify. Thanks again to everyone involved for your attention to this matter!

Thanks a lot for your suggestion! If you're interested in contributing this feature, @crazywoola would love to have a preliminary chat with you. 😉

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.