nos

nos copied to clipboard

- https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1 - https://huggingface.co/mistralai/Mixtral-8x7B-v0.1

outtanames

Default mins on cpu count and container mem not possible on M2 Macbook air, though most models will run with more relaxed limits. Let's just add this as a dedicated...

outtanames

spillai

https://github.com/autonomi-ai/nos/blob/main/docs/concepts/model-manager.md

spillai

https://github.com/autonomi-ai/nos/blob/main/docs/concepts/model-spec.md

spillai

Avoid having to generate the pb2 files by the client and instead generate pb files on `make dist` and add it to the wheel file.

spillai

spillai

Register Inference module directly from class definition: `RegisterModuleFromCls`

1

Ability to register module from class definition: (expose `ModuleFromCls` in `InferenceModule`).

spillai

[docs] Inference Runtime (ray + conda deps + HW deployment spec)

1

https://github.com/autonomi-ai/nos/blob/main/docs/concepts/runtime-environments.md

spillai

[docs] How to add a new `RuntimeEnv` (client + server)

1

spillai