ChatQnA readme issues

[tbykowsk]

Hi, I followed ChatQnA Application readme and encountered some problems. I started with code from the master branch.

1. In the step Start the Backend Service, any request to the streaming endpoint returns an error:

INFO:     Application startup complete.
	INFO:     Uvicorn running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
	INFO:     <IP>:50773 - "OPTIONS /v1/rag/chat_stream HTTP/1.1" 200 OK
	[rag - chat_stream] POST request: /v1/rag/chat_stream, params:{'query': 'What is the total revenue of Nike in 2023?"', 'knowledge_base_id': 'default'}
	[rag - chat_stream] history: []
	[rag - reload retriever] reload with index: rag-redis
	INFO:     <IP>:50773 - "POST /v1/rag/chat_stream HTTP/1.1" 200 OK
	score_threshold is deprecated. Use distance_threshold instead.score_threshold should only be used in similarity_search_with_relevance_scores.score_threshold will be removed in a future release.
	Metadata key source not found in metadata. Setting to None.
	Metadata fields defined for this instance: ['source', 'start_index']
	[...]
	Metadata key start_index not found in metadata. Setting to None.
	Metadata fields defined for this instance: ['source', 'start_index']
	ERROR:    Exception in ASGI application
	Traceback (most recent call last):
	  File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 265, in __call__
	    await wrap(partial(self.listen_for_disconnect, receive))
	  File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap
	    await func()
	  File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 238, in listen_for_disconnect
	    message = await receive()
	              ^^^^^^^^^^^^^^^
	  File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 538, in receive
	    await self.message_event.wait()
	  File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
	    await fut
	asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f0a23abf1d0
	 
	During handling of the above exception, another exception occurred:
	 
	  + Exception Group Traceback (most recent call last):
	  |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
	  |     result = await app(  # type: ignore[func-returns-value]
	  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
	  |     return await self.app(scope, receive, send)
	  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  |   File "/home/user/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
	  |     await super().__call__(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
	  |     await self.middleware_stack(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
	  |     raise exc
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
	  |     await self.app(scope, receive, _send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
	  |     await self.simple_response(scope, receive, send, request_headers=headers)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
	  |     await self.app(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
	  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
	  |     raise exc
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
	  |     await app(scope, receive, sender)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
	  |     await self.middleware_stack(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
	  |     await route.handle(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
	  |     await self.app(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
	  |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
	  |     raise exc
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
	  |     await app(scope, receive, sender)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 75, in app
	  |     await response(scope, receive, send)
	  |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 258, in __call__
	  |     async with anyio.create_task_group() as task_group:
	  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
	  |     raise BaseExceptionGroup(
	  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
	  +-+---------------- 1 ----------------
	    | Traceback (most recent call last):
	    |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap
	    |     await func()
	    |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 252, in stream_response
	    |     chunk = chunk.encode(self.charset)
	    |             ^^^^^^^^^^^^
	    | AttributeError: 'NoneType' object has no attribute 'encode'
	    +------------------------------------

This error causes that the Frontend Service is unresponsive, because it connects to /v1/rag/chat_stream endpoint which is broken.

When I reverted app/server.py to this commit the streaming endpoint started to work. It would be useful to add information to the instruction about a commit/release it was validated with.

2. The Frontend Service allows to add new data sources Please upload your local file or paste a remote file link, and Chat will respond based on the content of the uploaded file. In the log new data sources are ingested correctly:

[rag - create] POST request: /v1/rag/create, filename:architecture-instruction-set-extensions-programming-reference.pdf
[rag - create kb folder] upload path: upload_dir/kb_0f19a9e4/upload_dir, persist path: upload_dir/kb_0f19a9e4/persist_dir
[rag - create] file saved to local path: upload_dir/kb_0f19a9e4/upload_dir/2024-04-17-16:54:11-architecture-instruction-set-extensions-programming-reference.pdf
[rag - create] starting to create local db...
[rag - create retriever] create with index: rag-rediskb_0f19a9e4
[nltk_data] Downloading package punkt to /home/user/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
`index_schema` does not match generated metadata schema.
If you meant to manually override the schema, please ignore this message.
index_schema: {'text': [{'name': 'content'}, {'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'vector': [{'name': 'content_vector', 'algorithm': 'HNSW', 'datatype': 'FLOAT32', 'dims': 768, 'distance_metric': 'COSINE'}]}
generated_schema: {'text': [{'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'tag': []}

[rag - create] kb created successfully
INFO:     <IP>:51563 - "POST /v1/rag/create HTTP/1.1" 200 OK

[rag - upload_link] POST request: /v1/rag/upload_link, link list:['https://en.wikipedia.org/wiki/American_white_pelican']
[rag - create kb folder] upload path: upload_dir/kb_2ce41686/upload_dir, persist path: upload_dir/kb_2ce41686/persist_dir
[rag - upload_link] starting to create local db...
start fetch %s... https://en.wikipedia.org/wiki/American_white_pelican
`index_schema` does not match generated metadata schema.
If you meant to manually override the schema, please ignore this message.
index_schema: {'text': [{'name': 'content'}, {'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'vector': [{'name': 'content_vector', 'algorithm': 'HNSW', 'datatype': 'FLOAT32', 'dims': 768, 'distance_metric': 'COSINE'}]}
generated_schema: {'text': [{'name': 'source'}, {'name': 'identify_id'}], 'numeric': [], 'tag': []}


[rag - upload_link] kb created successfully
INFO:     <IP>:20643 - "POST /v1/rag/upload_link HTTP/1.1" 200 OK

However, new data sources do not seem to be instantly included during response generation. A restart of app/server.py is required for new information to be available, even though in the backend an index changes with each uploaded document, and is reloaded with each response, example of log: [rag - reload retriever] reload with index: rag-rediskb_147637c0. To confirm this problem, I have used a working app/server.py from the aforementioned commit, the rest of code was from the master branch.

Please look into those issues. Thanks!