Cinnamon
[REQUEST] Built-in API
Reference Issues
#330
Summary
I would like to have a minimalistic API endpoints accessed at http://GRADIO_SERVER_NAME:GRADIO_SERVER_PORT/api.
- GET
version- to check current and latest available version - GET
files- list of all the files with their information - POST
generate- generation endpoint to send request from another app. It would need prompt and list of files as arguments - POST
upload- uploading files in the given format - (optional) PUT
settings- to change setting via external app - (optional) GET
users(need auth) - list of added users with their info
Basic Example
As a developer I like using such great tools as Kotaemon in my local Discord bots or other simple apps. It could allow me to create a bot which can get input from Discord user as a prompt to provided files in attachments.
Drawbacks
I do not know python a lot so implementation could be risky not to break existing features.
Additional information
It could potentially bring more contributors to the project if it offered own API.
Hi @238SAMIxD this is a popular request. Currently we are tracking the discussion at https://github.com/Cinnamon/kotaemon/discussions/330#discussioncomment-10858993
Overall it is possible to create API like you mentioned without breaking changes, but it will require some effort.
I've encountered this issue as well. Below is my solution, which allowed me to extend the API functionality and address the limitations of the Gradio API.
# app.py
from fastapi import FastAPI, Request # Import FastAPI and Request class
from fastapi.responses import JSONResponse # Import JSONResponse for API responses
import gradio as gr # Import Gradio for UI integration
import uvicorn # Import Uvicorn to run the FastAPI application
from ktem.main import App # Import Gradio application from ktem.main
# Utility function to find the index of an object in a list by matching a key-value pair
def index_of_obj(objects, key, value):
for index in objects:
if getattr(objects[index], key) == value: # Check if the object's key matches the value
return index
return -1 # Return -1 if no matching object is found
# Initialize key-index mapping for Gradio functions
def init_ktem_constants(demo):
func_names = ["list_file"] # List of function names to be mapped
func_indices = {} # Dictionary to store function indices
# Map each function name to its index in the Gradio app
for func_name in func_names:
func_indices[func_name] = index_of_obj(demo.fns, "name", func_name)
print("func_name:", func_name, "func_indices:", func_indices[func_name])
return func_indices # Return the function index map
# Initialize and extend the API with custom and Gradio routes
def init_extend_api(demo):
extendapi = FastAPI() # Create a new FastAPI instance
ktem_constants = init_ktem_constants(demo) # Initialize function index map
list_file_func_index = ktem_constants["list_file"] # Get the index for 'list_file' function
# Custom API route for testing
@extendapi.get("/extendapi/test")
async def get_test():
return JSONResponse(content={"status": True, "message": "Hello from FastAPI!"})
# Gradio API route to get a list of files
@extendapi.get("/extendapi/file")
async def get_extendapi_file(request: Request):
# TODO: Replace with actual user_id loading logic
user_id = 1
file_list = demo.fns[list_file_func_index].fn(user_id) # Call 'list_file' function with user_id
return {"status": True, "message": "Success", "file_list": file_list[0]}
return extendapi # Return the FastAPI instance with extended APIs
# Create an instance of the Gradio application
gradio_app = App().make() # Create the Gradio app from the custom App class
extendapi = init_extend_api(gradio_app) # Initialize the extended API
# Mount Gradio interface into FastAPI under the root path "/"
gr.mount_gradio_app(
extendapi,
gradio_app,
path="/", # Set the path for Gradio app
)
# Run FastAPI application with Gradio interface
if __name__ == "__main__":
uvicorn.run(extendapi, host="0.0.0.0", port=7860) # Launch the app on port 7860
To be honest I do not want to update the code. I would prefer to have it natively to run the docker container. Any update could interfere with it. This is why the most needed feature for me is to have version check so I can add it to my script to automatically update the docker image. Thank you for your responses
Thanks for your feedback. We are already working on it and will update the progress soon.
I've encountered this issue as well. Below is my solution, which allowed me to extend the API functionality and address the limitations of the Gradio API.
# app.py from fastapi import FastAPI, Request # Import FastAPI and Request class from fastapi.responses import JSONResponse # Import JSONResponse for API responses import gradio as gr # Import Gradio for UI integration import uvicorn # Import Uvicorn to run the FastAPI application from ktem.main import App # Import Gradio application from ktem.main # Utility function to find the index of an object in a list by matching a key-value pair def index_of_obj(objects, key, value): for index in objects: if getattr(objects[index], key) == value: # Check if the object's key matches the value return index return -1 # Return -1 if no matching object is found # Initialize key-index mapping for Gradio functions def init_ktem_constants(demo): func_names = ["list_file"] # List of function names to be mapped func_indices = {} # Dictionary to store function indices # Map each function name to its index in the Gradio app for func_name in func_names: func_indices[func_name] = index_of_obj(demo.fns, "name", func_name) print("func_name:", func_name, "func_indices:", func_indices[func_name]) return func_indices # Return the function index map # Initialize and extend the API with custom and Gradio routes def init_extend_api(demo): extendapi = FastAPI() # Create a new FastAPI instance ktem_constants = init_ktem_constants(demo) # Initialize function index map list_file_func_index = ktem_constants["list_file"] # Get the index for 'list_file' function # Custom API route for testing @extendapi.get("/extendapi/test") async def get_test(): return JSONResponse(content={"status": True, "message": "Hello from FastAPI!"}) # Gradio API route to get a list of files @extendapi.get("/extendapi/file") async def get_extendapi_file(request: Request): # TODO: Replace with actual user_id loading logic user_id = 1 file_list = demo.fns[list_file_func_index].fn(user_id) # Call 'list_file' function with user_id return {"status": True, "message": "Success", "file_list": file_list[0]} return extendapi # Return the FastAPI instance with extended APIs # Create an instance of the Gradio application gradio_app = App().make() # Create the Gradio app from the custom App class extendapi = init_extend_api(gradio_app) # Initialize the extended API # Mount Gradio interface into FastAPI under the root path "/" gr.mount_gradio_app( extendapi, gradio_app, path="/", # Set the path for Gradio app ) # Run FastAPI application with Gradio interface if __name__ == "__main__": uvicorn.run(extendapi, host="0.0.0.0", port=7860) # Launch the app on port 7860
Very good, if I want to implement the chat function through an API, how should I write the code? @jerry2971
@a652 First, you can use ?view=api to access the Gradio API and use the API Recorder to generate the code. If the Gradio API does not meet your needs, then you can consider using other solutions.
In my approach, you'll need to add the API name to func_names and implement the corresponding parameter interface.
Can someone provide the exact Gradio API name for file upload? When I look at all the api end points in gradio there are multiple and none of them are working for file upload. For e.g., index_fn etc. It would be great if there was a POST API end point but at this point I am happy to just get a gradio working API where i can submit the file list to load and get the response back on the status.
