big indexing - openai requests may break

[mvpmaster]

when i tried to do index many big documents on default mode (openai),

at some point completions request fail on 404 nginx error and fail whole script, and it was not able to save current state

File "/Desktop/projects/PathRAG/PathRAG/operate.py", line 317, in _process_single_content glean_result = await use_llm_func(continue_prompt, history_messages=history) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Desktop/projects/PathRAG/PathRAG/utils.py", line 119, in wait_func result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "Desktop/projects/PathRAG/PathRAG/llm.py", line 483, in gpt_4o_mini_complete return await openai_complete_if_cache(

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 404 Not Found"

so i started from beginning and making backup of index after each document processing (but then wrote some fast fix code)

gpt_4o_mini_complete - need check , if something go wrong