Added streaming output support for local search & global search
Description
This PR introduces changes to enable streaming output in the LLM's generate method, as well as in the search methods of both LocalSearch and GlobalSearch classes. Intended to facilitate web application development or any other scenarios where streaming and real-time display of generated content is required.
Related Issues
none
Proposed Changes
- Updated the
generatemethod's return type toGenerator[str, None, str]ofBaseLLMandChatOpenAIand implement it. - Updated the
searchmethod's return type toGenerator[str, None, SearchResult]ofLocalSearch,GlobalSearchandBaseSearch, and implement it. - Added a '--streaming' flag to the cli, allows users to specify whether they want to enable streaming output for local and global search.
Checklist
- [x] I have tested these changes locally.
- [x] I have reviewed the code changes.
- [x] I have updated the documentation (if necessary).
- [x] I have added appropriate unit tests (if applicable).
Additional Notes
A caveat with these changes is that when a method contains a yield statement, the Python interpreter automatically defines it as a generator function, meaning that it always returns a generator. If streaming is not needed, it becomes less straightforward to directly retrieve the final result, as you must use a try-except block to catch the StopIteration exception and extract the value from it. This can lead to less elegant code.
If necessary, the streaming functionality could be refactored into separate methods to maintain clarity and separation of concerns.