block
Canonical models for Providers
Build core canonical model implementation
Two scripts
- build_canonical_model -> pulls all models from openrouter and loads them into our json file
- canonical_model_checker -> produces the full mappings from our providers as to which (provider, models) are being mapped into which canonical models.
Providers can use map_to_canonical_model to lookup this info (if the model is mapped).
Once we're settled on this will further integrate it in providers and pull + add more information to these.
Currently integrated in model filtering.
Pricing is next; The hit rate for the current system is pretty low outside of openrouter.
nice - @katzdave how would you like this to be tested/reviewed, as a lot of new stuff
nice - @katzdave how would you like this to be tested/reviewed, as a lot of new stuff
Mostly just if this is the general abstraction we want? Its all pretty standalone right now map_to_canonical_model will let you pull the data from within the provider. Otherwise canonical models holds the current state (you can update it via re-running build_canonical_models).
The mapping report isn't strictly necessary, but lets you see the diff between how existing models got mapped so you can audit changes to the canonical models json more carefully.
Thinking we get some consensus on what we want to store in the repo and then can do the integration across more providers incrementally