rig icon indicating copy to clipboard operation
rig copied to clipboard

feat: Allow for verification of the models list for each providers

Open ProHaller opened this issue 7 months ago • 4 comments

  • [x] I have looked for existing issues (including closed) about this

This has been brought up in #485

Feature Request

High-level description

Add a structured and testable representation of provider model lists, replacing the current const-based implementation with enums (optionally via strum for easy string conversion) to allow for better integration with API responses, especially /models GET method to list available models.

Motivation

Currently, models like xAI’s are represented as a list of const &str entries:

pub const GROK_2_1212: &str = "grok-2-1212";
// ...

This format:

  • Cannot be iterated over easily
  • Is disconnected from dynamic data like API [openai_compliant_provider]/v1/models responses
  • Cannot be used in a type-safe way to validate support or identify missing/deprecated models

Having a testable structure (e.g., an enum) would enable:

  • Easy comparison between hardcoded and fetched model lists
  • Compile-time validation where possible
  • Better integration in code paths that need to switch on model types
  • Lower risk of discrepancies between supported and documented models

Proposal

Replace the flat const model list with an enum, e.g.:

#[derive(EnumString, AsRefStr, EnumIter, Debug, Clone, PartialEq, Eq)]
#[strum(serialize_all = "kebab-case")]
pub enum XaiModel {
    #[strum(serialize = "grok-2-1212")]
    Grok21212,
    Grok2Vision1212,
    Grok3,
    Grok3Fast,
    Grok3Mini,
    Grok3MiniFast,
    Grok2Image1212,
}

This would allow:

  • Serialization to/from string (AsRefStr)
  • Easy iteration (EnumIter)
  • Better test integration (comparing enum list with API result)
  • Extensibility if future metadata (e.g., context length) needs to be embedded

A helper like XaiModel::iter().map(|m| m.as_ref()).collect::<Vec<_>>() makes it trivial to test against the actual response.

ProHaller avatar Jun 08 '25 00:06 ProHaller

It looks like there is some inherent difficulty in trying to obtain some of the model names without an API key.

Examples:

  • OpenAI docs
  • xAI docs - It doesn't specifically say in the endpoint that you need to be authenticated, but when you scroll to the top it says all endpoints are authed only.

We will probably have to build a service that does this automatically then opens a GitHub request to update new models (if there are any).

joshua-mo-143 avatar Jun 08 '25 14:06 joshua-mo-143

Oh. That's a very good point I did not consider. Especially since some services unlock high-end models based on the spending history. Not even talking about fine tuned models made available to specific organization.

Maybe in a first time make a manually checked base models enum with an Other(String) variant?

Or a dynamically created list created with client initialization?

ProHaller avatar Jun 09 '25 00:06 ProHaller

I think those are two different concepts, you can have a list of baked models at build time and another generic way to fetch an up to date list at runtime.

I don't think const are a problem per say as it makes it clear they are not meant to be iterated on.

Sytten avatar Jul 31 '25 15:07 Sytten