Integration: Bytez Chat Model Provider

[inf3rnus]

trafficstars

Description

Hi all! :wave:

We're Bytez, the largest model provider on the internet! We may also be one of the cheapest if not the cheapest.

We'd love to integrate with PortKey. Please see the changed files and let me know if anything needs to change.

I'd like to point out that we do a check against our api to see if a model is a "chat" model. This is stored in a simple cache that is just an object. If that's going to be a problem due to having an unbounded ceiling in terms of memory utilization pls lmk, and I will convert it to a LRU with 100 entries.

Our API's input signature is a bit more bespoke than other model providers, please lmk if the custom requestHandler I have is sufficient, or if there's an easier way to do what I've done.

Bonus feedback: Ya'll need an integration guide! Would be immensely useful :smile:

Motivation

We'd love to be integrated in PortKey!

Type of Change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Documentation update
• [ ] Refactoring (no functional changes)

How Has This Been Tested?

• [ ] Unit Tests
• [ ] Integration Tests
• [ x] Manual Testing

Screenshots (if applicable)

Checklist

• [ x] My code follows the style guidelines of this project
• [ x] I have performed a self-review of my own code
• [ x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ x] My changes generate no new warnings
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes

Related Issues