caikit
caikit copied to clipboard
caikit
Caikit is an AI toolkit that enables users to manage models through a set of developer friendly APIs.
## Is your feature request related to a problem? Please describe. I would like to stream raw bytes data instead of just filename references. ## Describe the solution you'd like...
## Description Even though we have this file compatibility_checker.py https://github.com/caikit/caikit/blob/main/caikit/runtime/service_generation/compatibility_checker.py, and the method `get_fields_for_message` function is used in generating RequestMessage [here](https://github.com/caikit/caikit/blob/03bcc3d273a692db24ffe6ddb04478394aafa9e4/caikit/runtime/service_generation/rpcs.py#L402), `ApiFieldNames().service_pb2_modules` is never actually assigned from a previous version....
## Is your feature request related to a problem? Please describe. Some tasks may require different resource requirements than others. For example, prompt tuning a small model may require 1...
## Is your feature request related to a problem? Please describe. It's possible to boot up a caikit runtime, and then run into CORS problems when using the http server....
## Description I want to be able to load HF models as caikit modules, without needing a caikit-style config.yml. Some code for this already exists, we just need to get...
## Describe the bug Figure out what all dependencies we need to pull from `fastapi` in order to support our REST server work and see if we can reduce the...
## Description As a caikit user, I want to be able to see docstrings on the OpenAPI spec so that I can point other users to the spec instead of...
## Is your feature request related to a problem? Please describe. As a user of `caikit.runtime`, I need to be able to know what happens when an inference call for...
## Is your feature request related to a problem? Please describe. Currently, the HTTP server [only supports unary/unary and unary/stream task inference](https://github.com/caikit/caikit/blob/main/caikit/runtime/http_server.py#L238). We need to support `stream/unary` and `stream/stream` as...
## Describe the bug With the `extra_forbid` flag set in all REST APIs, it makes it impossible to send in any field that's not part of the API signature. What...