nos
nos copied to clipboard
autonomi-ai
⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.
## Checks - [ ] `make lint`: I've run `make lint` to lint the changes in this PR. - [ ] `make test`: I've made sure the tests (`make test-cpu`...
The automatic delete after upload seems not working when using relative path.
Ability to modify Python in-place and update/restart the server on the fly
For clients that are connecting to an existing server, we'll need to verify if the server is consistent before attempting to pull the new server or connect to the server.
Custom models should be able to automatically get model info and infer that they're exposing a streaming method. We should be able to avoid having the user to specify `_stream=True`...
Load balancing, DDOS protection for public endpoints to expensive instances, etc.
Current deployments rely on streamlink. Customer request to support Webcasts directly.
In order to let users build slimmer images, we need to offer a slimmer base image with NOS internals and avoid bloating the server with all kinds of dependencies that...
Metadata
Owner
Metadata
⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.