inference
inference copied to clipboard
ENH: Add old ui as example
Rewrote deprecated Gradio UI with RESTful Client and added it to examples as gradio_arena.py
.
Instructions should be available by running the file with the --help
tag.
e.g. python path/to/inference/examples/gradio_arena.py --help
Here are the instructions, in case anyone needs it:
instructions to run:
1. Install Xinference, Llama-cpp-python, and other dependencies if necessary
2. Run command `xinference --host "localhost"` in terminal
3. You should see something similar to the following output:
INFO:xinference:Xinference successfully started. Endpoint: http://localhost:9997
INFO:xinference.core.service:Worker 127.0.0.1:21561 has been added successfully
INFO:xinference.deploy.worker:Xinference worker successfully started.
4. In the output, locate the endpoint. In the above case it is `http://localhost:9997`
5. Run this python file in new terminal window, change the endpoint accordingly
example run command (feel free to copy):
python gradio_arena.py \
--endpoint http://localhost:9997
Please also add this example to the docs: https://inference.readthedocs.io/en/latest/examples/index.html
You need to create a new rst file in this folder and add it to the index's tree of contents.