kaleidoscope
kaleidoscope copied to clipboard
VectorInstitute
A user toolkit for analyzing and interfacing with Large Language Models (LLMs)
# PR Type ([Feature | Fix | Documentation | Test]) Feature ## Short Description 1. Enabled Accelerate DeepSpeed for multi-GPU 2. Bumped up GPUs for both falcon versions (7B-2, 40B-4)...
# PR Type (Feature) ## Short Description Async batch inference ## Tests Added ...
**Describe the bug** When a model job fails a health check, it moves immediately into a FailedState and hence the gateway never sends the request to shut the job down....
# PR Type ([Feature | Fix | Documentation | Test]) ## Short Description ... ## Tests Added ...
**Describe the bug** If a service/container fails it should restart. **To Reproduce** Hit OOM on the web-server **Expected behavior** Web-server should restart and recover
**Issue:** The POST "/models/instances" endpoint will accept any string and return what looks like a valid model instance. **Expected Behaviour:** Return an error if the model name string does not...
Metadata
Owner
Metadata
A user toolkit for analyzing and interfacing with Large Language Models (LLMs)