Using OpenRouter triggers an exception because returned 'usage' parameters do not match expected structure

[johnpccd]

Using OpenRouter endpoints works well when added as an OpenChat model but the calculation of usage in inference() causes an exception since OpenRouter doesn't return the expected 'usage' parameters in the expected format. When that exception is handled, the text_output of the LLM is set to "/ERROR/"

Sample code to use OpenRouter:

model_name = 'google/gemini-pro'
os.environ["USER_MANAGED_OPEN_CHAT_API_KEY"] = os.environ.get("OPERNROUTER_API_KEY")
ModelRegistry().add_open_chat_model(model_name, api_base=os.environ.get('https://openrouter.ai/api/v1'))

Reference: OpenRouter documentation

Location to fix: https://github.com/llmware-ai/llmware/blob/ebd08a29b3e6bb21661dd56d08d3ed97fc01da5d/llmware/models.py#L1128

[doberst]

Thanks- this is great feedback - and appreciate the details and documentation - we will get this fixed in the next 1-2 days - will update you once its done - would be great for you to test ....

[doberst]

Added a fix - just merged in the main branch. It should avoid triggering exception when calling OpenRouter (although usage data will need to be gathered separately from OpenRouter API) ... Let us know if this solves the issue.

[doberst]

The change was implemented - no follow-up issues reported - so marking as closed. If you still run into any isues, please raise a new issue - or join us on Discord and share the details there.