sample-app-aoai-chatGPT icon indicating copy to clipboard operation
sample-app-aoai-chatGPT copied to clipboard

App "Generating" in the background at start when history is enabled, no issue if disabled

Open Francois2024 opened this issue 10 months ago • 7 comments

Describe the bug If I enable the history option, the app tries to generate content when the url is loaded when starting the app. This prevents the user from using the chatbot for 5-10 seconds. No content is generated in the end but there is a process running in the background which shows the app as being in content generation mode.

To Reproduce Steps to reproduce the behavior:

  1. Load your app with history option enabled
  2. "Stop Generating" is active for 5-10 sec (eventhough no content has been entered in the chat window)
  3. Wait 5-10 sec and then user can start prompting the chatbot

Expected behavior Same behaviour as when history is not enabled. No content generation attempt at start.

Screenshots If applicable, add screenshots to help explain your problem.

Configuration: Please provide the following

  • GPT4 Turbo (1106 preview)
  • Chat history enabled
  • No changes to the original repo (except for some visual css changes).

Additional context Add any other context about the problem here.

Francois2024 avatar Apr 27 '24 18:04 Francois2024

There's no content actually being generated. In the first few seconds the app connects to the history database and reads your chats from there. Until these requests have completed, the application shows the "generating" hint. That's also the reason this behavior only occurs when history is enabled.

neissljanik avatar May 07 '24 09:05 neissljanik

Thanks for the explanation. Is there a way to disable this behavior so it does not read the chats history? This is not useful for us and it creates a 5-10 seconds loading delay before the user can query the app.

Francois2024 avatar May 08 '24 07:05 Francois2024

Sort of, but not without some programming knowledge. We implemented lazy loading for chat messages in our fork, which helps to reduce the number of requests sent initially and therefore cuts down on loading time. However, if you want to keep the history feature, there's no sensible way around establishing a connection to the database initially. You can't save messages to the DB if there's no connection to the DB.

neissljanik avatar May 08 '24 08:05 neissljanik

Ok got it thanks for your help. We will look into it.

Francois2024 avatar May 08 '24 08:05 Francois2024

We removed the history feature for now until we find a solution.

Francois2024 avatar May 08 '24 08:05 Francois2024

FWIW, our fork currently has 4.5s of loading time, but we're doing some additional requests as well, so implementing lazy loading for the chat messages would probably reduce the initial wait to 3-4s. Keep in mind that this is a very rough estimation influenced by a boatload of factors.

neissljanik avatar May 08 '24 08:05 neissljanik

Ok! Thanks. This repo is by the way very useful as a starting point.

Francois2024 avatar May 08 '24 08:05 Francois2024

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Jul 08 '24 01:07 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Jul 22 '24 01:07 github-actions[bot]