Alex Cheema
Alex Cheema
await self.inference_engine.infer_tensor(request_id, shard, tensor, inference_state=inference_state)
Hey! Thanks for making the issue. Could use a bit more info to debug this. What devices are you running on?
Does it work when you pip install llvmlite?
As far as I can tell, tailscale==0.6.1 does indeed exist. Install works fine for me and in CI. See: https://pypi.org/project/tailscale/#history
``` (.venv) alex@Alexs-MBP exo % pip show tailscale Name: tailscale Version: 0.6.1 Summary: Asynchronous client for the Tailscale API. Home-page: https://github.com/frenck/python-tailscale Author: Franck Nijhof Author-email: [email protected] License: MIT Location: /Users/alex/exo/.venv/lib/python3.12/site-packages...
Should be fixed now. Can you test again on latest main @MiguelSanzBr @FFAMax
Exo provides a ChatGPT-compatible API endpoint on each device. Langchain supports ChatGPT therefore it should be possible to swap in exo. If you run into any issues or bugs please...
I'd love to spark some discourse about this. I've been thinking about similar ideas. I guess one of the immediate questions that comes to mind is **privacy**. If you're sending...
Almost what I envisioned - only thing I would change is to preload after the preemptive download. We don't want to download all possible model shards, only the relevant one.
Merged! Certainly worthy of a $200 retrospective bounty. Please email [email protected] to claim the bounty!
It requires installing pipx though right? At this point we might as well have an install script. I was thinking we need a master install script that can install "hidden"...