mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Integrating Mistral.rs with Swiftide

Open timonv opened this issue 4 months ago • 0 comments

Hey there,

Figured I'd give it another shot to integrate mistral.rs with Swiftide. Tested several text models and it works great. Kudos on the new v4 builder style api, it's very clean 👍

However, github actions is now running into limitations and kills the worker with a 143 (afaik this is usually due to high cpu).

I've been looking at cargo timings and tried some tuning, but no luck so far. Does mistral.rs have to do tuning to be able to run in GA? I couldn't find anything specific. The full PR is at https://github.com/bosun-ai/swiftide/pull/386

Side note: Would it be possible to publish mistralrs as a crate?

timonv avatar Oct 12 '24 12:10 timonv