alpaca-lora Something went wrong Unexpected token '<', "<html> <h"... is not valid JSON

When using CPU as a device, after trying various prompts, gradio returns with the following error when running generate.py:


Something went wrong
Unexpected token '<', "<html> <h"... is not valid JSON

Interestingly I do not see any errors in the console log, it seems to be a formatting issue with the gradio ui.

the prompts I tried:

Write a Python program that prints the first 10 Fibonacci numbers.
write python code that renames all files starting with "mleml" in the current folder to "kek".

Mar 18 '23 12:03 ThatCoffeeGuy

could it be a server timeout ? how long did the task take ?

Mar 20 '23 14:03 norsigmacom

Reasonable time, about 4-5 seconds, I doubt it'd be a timeout.

Mar 22 '23 10:03 ThatCoffeeGuy

Change run type type as GPU and try.

Mar 22 '23 16:03 Anuvathan

Change run type type as GPU and try.

I don't have enough VRAM for the larger models, hence why trying with CPU.

Mar 22 '23 16:03 ThatCoffeeGuy

demo = gr.Interface( fn=process_document, inputs="image", outputs="json", # live = True, title=" OCR", enable_queue=True, cache_examples=False, debug=True, ) demo.launch(share=True)

This is my gradio API function code. Previously I had the same issue. But after I change to GPU in colab It's working.

and it's not for this repository. It's for other different type.

Mar 22 '23 16:03 Anuvathan

https://github.com/gradio-app/gradio/issues/3185 change demo.launch(share=True) to demo.queue().launch(share=True) Enabling the queue is required for inference times > 60 seconds: https://gradio.app/key-features/#queuing

Apr 16 '23 00:04 DanqingZ

gradio-app/gradio#3185 change demo.launch(share=True) to demo.queue().launch(share=True) Enabling the queue is required for inference times > 60 seconds: https://gradio.app/key-features/#queuing

fix my same problem

Apr 23 '23 09:04 zychyz95822

I have changed to demo.queue().launch(share=True), but still cannot fix my problem

Jun 05 '23 09:06 18620146591

gradio-app/gradio#3185 추론 시간 > 60초에 대기열 활성화가 필요함 으로 변경 : https://gradio.app/key-features/#queuing demo.launch(share=True) demo.queue().launch(share=True)

thanks for your comments

Jun 25 '23 15:06 chanheehi

Hi @DanqingZ, I'm running gradio Application with FastAPI, how to handle it in such cases?

Following is the configuration of my gradio application:

from fastapi import FastAPI
import gradio as gr

app = FastAPI()
app = gr.mount_gradio_app(app, demo, path="/my_custom_route")

Getting following error:

Connection errored out.

How to handle requests which take more than 60s in case of using gradio with FastAPI?

Aug 24 '23 12:08 swapnil3597

Hi @DanqingZ, I'm running gradio Application with FastAPI, how to handle it in such cases?

Following is the configuration of my gradio application:
from fastapi import FastAPI
import gradio as gr

app = FastAPI()
app = gr.mount_gradio_app(app, demo, path="/my_custom_route")
Getting following error:
Connection errored out.
How to handle requests which take more than 60s in case of using gradio with FastAPI?

I am experiencing the same issue. I tried passing in the gradio instance as queue: app = gr.mount_gradio_app(app, demo.queue(), path="/my_custom_route") but is not working

Sep 10 '23 03:09 arditobryan

alpaca-lora alpaca-lora copied to clipboard

Something went wrong Unexpected token '<', "<html> <h"... is not valid JSON

alpaca-lora
alpaca-lora copied to clipboard