Live interface does not update for every change of textbox

[freddyaboulton]

Describe the bug

If you have a live interface, the predict function sometimes is not run for every update of the component - especially if the predict function has some delay and the updates happen faster than the runtime of the predict function.

You can repro with this interface

import gradio as gr
import time

def print_with_delay(x):
    time.sleep(0.2)
    return x

gr.Interface(print_with_delay, "text", "text", live=True).launch()

Note that I typed "freddy" but the text box only shows "fred".

A more realistic example is https://huggingface.co/spaces/adirik/OWL-ViT , where running with live=True will make the object detection query not contain all the key words.

https://user-images.githubusercontent.com/41651716/183110769-c907cc8b-6c83-48ac-b472-5b6b6e241cfa.mp4

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Screenshot

No response

Logs

-

System Info

3.1.3

Severity

serious, but I can work around it

[abidlabs]

Yup! The problem is that is a new prediction isn't sent until the previous one is complete, which can create a very "non-live" experience, especially on Spaces and Colab.

I know we considered adding a debounce to the Textbox and Slider components for this reason. Should we add it here @pngwn @aliabid94?

[pngwn]

We did have a debounce but we removed it because it caused some bugs. https://github.com/gradio-app/gradio/pull/932

I think the correct solution here is to send all requests and ensure the most recent request/ action takes priority over earlier ones. This will require some changes to how we are processing request both on the frontend and the backend.

However, this could cause issues with applications/ updates that are stateful. So realistically it would be better to process all requests in order (batching them where possible).

In general with live updates we need to reduce latency as much as possible, using websockets for everything would provide the technical foundation for this kinda of responsiveness. Something I mentioned recently on slack.

[pngwn]

Copied from response in slack:

Live updates need to be looked at holistically anyway imo, its a really important usecase but Gradio doesn't really do a great job right now because of how we process requests, and how we handle loading states. Queueing also introduces some additional challenges here, at least from a design PoV.

Loading states in general are pretty rudimentary right now and only really support the classic action -> run prediction -> wait -> display flow. They aren't suited to these kinds of real time updates or for streamed updates. None of these issues are impossible to resolve, they just require some thought and a little engineering effort.

[abidlabs]

This should be solved with #2436 so moving this to the 4.0 milestone

[abidlabs]

Another case where this pops up is #3627, as reported by @hysts so I've renamed this issue to reflect the more broader issue at hand

[picobyte]

Besides a change event, a Textbox blur (loss of focus) event is also handled after e.g. a button click, which seems incorrect. I'd say it makes more sense for a Textbox input, change and blur to be handled in that order, before the handling of another second component. I believe this issue is related? Is the cause async handling or an incorrect order of events?

A workaround is passing all components as input with the button press, but makes the handling more cumbersome.

Edit: to answer my own question, it seems to be an async issue. the problem appears to go away if the blur event does nothing that stalls (non-io).

[abidlabs]

Fixed in v4 branch, thanks @dawoodkhan82!