KruxAI
Cannot run the app due to OpenAI rate limit breach
Excited to try out Ragbuilder, but I can't get it to run successfully due to an OpenAI rate issue ...
[INFO] 2024-08-26 16:01:41 - common.py - raise self._make_status_error_from_response(err.response) from None
[INFO] 2024-08-26 16:01:41 - common.py - openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-14onX0p3Xh0qpWBmqQ5WcBUO on tokens per min (TPM): Limit 30000, Used 29260, Requested 1564. Please try again in 1.648s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
I am only using one PDF to test.
Here are the full logs ...
[INFO] 2024-08-26 15:31:56 - common.py - USER_AGENT environment variable not set, consider setting it to identify your requests.
[INFO] 2024-08-26 15:32:04 - common.py - USER_AGENT environment variable not set, consider setting it to identify your requests.
[INFO] 2024-08-26 15:32:05 - ragbuilder.py - LOG_FILENAME = /Users/matthewharris/Desktop/git/ragbuilder/logs/2024-08-26_15-31-55.log
[INFO] 2024-08-26 15:32:06 - common.py - INFO: Started server process [40485]
[INFO] 2024-08-26 15:32:06 - common.py - INFO: Waiting for application startup.
[INFO] 2024-08-26 15:32:06 - common.py - INFO: Application startup complete.
[INFO] 2024-08-26 15:32:06 - common.py - INFO: Uvicorn running on http://0.0.0.0:8005 (Press CTRL+C to quit)
[INFO] 2024-08-26 15:32:07 - common.py - INFO: 127.0.0.1:64202 - "GET / HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:32:07 - common.py - INFO: 127.0.0.1:64202 - "GET /static/main.js HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:32:08 - common.py - INFO: 127.0.0.1:64202 - "GET /static/ragbuilder_light.png HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:32:08 - common.py - INFO: 127.0.0.1:64202 - "GET /favicon.ico HTTP/1.1" 404 Not Found
[INFO] 2024-08-26 15:32:19 - common.py - INFO: 127.0.0.1:64216 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:33:30 - common.py - INFO: 127.0.0.1:64221 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:33:36 - common.py - INFO: 127.0.0.1:64223 - "GET /templates HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:33:36 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:33:36 - common.py - INFO: 127.0.0.1:64222 - "POST /check_test_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:34:57 - ragbuilder.py - enable_analytics = True
[INFO] 2024-08-26 15:34:57 - ragbuilder.py - Initiating parsing config: {'description': 'WFP RAG', 'sourceData': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf', 'compareTemplates': True, 'includeNonTemplated': True, 'selectedTemplates': ['query_rewrite', 'step_back_prompt', 'semantic_chunker', 'hyde', 'hybrid_rag', 'rrf'], 'chunkingStrategy': {'MarkdownHeaderTextSplitter': True, 'HTMLHeaderTextSplitter': True, 'SemanticChunker': True, 'RecursiveCharacterTextSplitter': True, 'CharacterTextSplitter': True}, 'chunkSize': {'min': 500, 'max': 2000}, 'embeddingModel': {'OpenAI:text-embedding-3-small': False, 'OpenAI:text-embedding-3-large': True, 'OpenAI:text-embedding-ada-002': False, 'HuggingFace': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceEmbeddingModel': '', 'azureOAIEmbeddingModel': '', 'googleVertexAIEmbeddingModel': '', 'ollamaEmbeddingModel': '', 'vectorDB': 'chromaDB', 'retriever': {'vectorSimilarity': True, 'vectorMMR': True, 'bm25Retriever': True, 'multiQuery': True, 'parentDocFullDoc': True, 'parentDocLargeChunk': True}, 'topK': {'search_k_5': True, 'search_k_10': False, 'search_k_20': False}, 'contextualCompression': True, 'llm': {'OpenAI:gpt-4o-mini': True, 'OpenAI:gpt-4o': False, 'OpenAI:gpt-3.5-turbo': False, 'OpenAI:gpt-4-turbo': False, 'HuggingFace': False, 'Groq': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceLLMModel': '', 'groqLLMModel': '', 'azureOAILLMModel': '', 'googleVertexAILLMModel': '', 'ollamaLLMModel': '', 'generateSyntheticData': True, 'optimization': 'bayesianOptimization', 'evalFramework': 'RAGAS', 'evalEmbedding': 'OpenAI:text-embedding-3-large', 'evalLLM': 'OpenAI:gpt-4o-mini', 'sotaEmbeddingModel': 'OpenAI:text-embedding-3-large', 'sotaLLMModel': 'OpenAI:gpt-4o-mini', 'compressors': {'LongContextReorder': True, 'CrossEncoderReranker': True, 'EmbeddingsRedundantFilter': False, 'EmbeddingsClusteringFilter': False, 'LLMChainFilter': False}, 'syntheticDataGeneration': {'testSize': '20', 'criticLLM': 'OpenAI:gpt-4o', 'generatorLLM': 'OpenAI:gpt-4o', 'generatorEmbedding': 'OpenAI:text-embedding-3-large'}, 'testDataPath': None, 'existingSynthDataPath': None, 'testSize': None, 'criticLLM': None, 'generatorLLM': None, 'generatorEmbedding': None, 'numRuns': '50'}
[INFO] 2024-08-26 15:34:57 - ragbuilder.py - Disabled options: ['EmbeddingsRedundantFilter', 'EmbeddingsClusteringFilter', 'LLMChainFilter', 'OpenAI:gpt-4o', 'OpenAI:gpt-3.5-turbo', 'OpenAI:gpt-4-turbo', 'HuggingFace', 'Groq', 'AzureOAI', 'GoogleVertexAI', 'Ollama', 'search_k_10', 'search_k_20', 'OpenAI:text-embedding-3-small', 'OpenAI:text-embedding-ada-002', 'HuggingFace', 'AzureOAI', 'GoogleVertexAI', 'Ollama']
[INFO] 2024-08-26 15:34:57 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:34:57 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:34:57 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:34:57 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:34:57 - embedding.py - getEmbedding Invoked
[INFO] 2024-08-26 15:34:57 - embedding.py - OpenAIEmbeddings Invoked: OpenAI:text-embedding-3-large
[INFO] 2024-08-26 15:34:57 - generate_data.py - Loading docs...
[INFO] 2024-08-26 15:34:57 - loader.py - ragbuilder_loader Invoked:{'input_path': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf', 'return_code': False}
[INFO] 2024-08-26 15:34:57 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:34:57 - loader.py - Source type identified: directory
[INFO] 2024-08-26 15:34:57 - loader.py - ragbuilder_directory_loader_exec Invoked
[ERROR] 2024-08-26 15:35:15 - loader.py - Error in ragbuilder_directory_loader_exec: Detected a JSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output.
[ERROR] 2024-08-26 15:35:15 - generate_data.py - Error loading docs for synthetic test data generation.
[ERROR] 2024-08-26 15:35:15 - ragbuilder.py - Synthetic test data generation failed.
[INFO] 2024-08-26 15:35:15 - common.py - INFO: 127.0.0.1:64224 - "POST /rbuilder HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:36:56 - common.py - INFO: 127.0.0.1:64260 - "GET / HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:36:56 - common.py - INFO: 127.0.0.1:64260 - "GET /static/main.js HTTP/1.1" 304 Not Modified
[INFO] 2024-08-26 15:36:56 - common.py - INFO: 127.0.0.1:64260 - "GET /static/ragbuilder_light.png HTTP/1.1" 304 Not Modified
[INFO] 2024-08-26 15:37:04 - common.py - INFO: 127.0.0.1:64262 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:37:15 - common.py - INFO: 127.0.0.1:64263 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:37:19 - common.py - INFO: 127.0.0.1:64265 - "GET /templates HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:37:19 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:37:19 - common.py - INFO: 127.0.0.1:64264 - "POST /check_test_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:37:41 - ragbuilder.py - enable_analytics = True
[INFO] 2024-08-26 15:37:41 - ragbuilder.py - Initiating parsing config: {'description': 'WFP QA', 'sourceData': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf_only', 'compareTemplates': True, 'includeNonTemplated': True, 'selectedTemplates': ['query_rewrite', 'step_back_prompt', 'semantic_chunker', 'hyde', 'hybrid_rag', 'rrf'], 'chunkingStrategy': {'MarkdownHeaderTextSplitter': True, 'HTMLHeaderTextSplitter': True, 'SemanticChunker': True, 'RecursiveCharacterTextSplitter': True, 'CharacterTextSplitter': True}, 'chunkSize': {'min': 500, 'max': 2000}, 'embeddingModel': {'OpenAI:text-embedding-3-small': False, 'OpenAI:text-embedding-3-large': True, 'OpenAI:text-embedding-ada-002': False, 'HuggingFace': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceEmbeddingModel': '', 'azureOAIEmbeddingModel': '', 'googleVertexAIEmbeddingModel': '', 'ollamaEmbeddingModel': '', 'vectorDB': 'chromaDB', 'retriever': {'vectorSimilarity': True, 'vectorMMR': True, 'bm25Retriever': True, 'multiQuery': True, 'parentDocFullDoc': True, 'parentDocLargeChunk': True}, 'topK': {'search_k_5': True, 'search_k_10': False, 'search_k_20': False}, 'contextualCompression': True, 'llm': {'OpenAI:gpt-4o-mini': True, 'OpenAI:gpt-4o': False, 'OpenAI:gpt-3.5-turbo': False, 'OpenAI:gpt-4-turbo': False, 'HuggingFace': False, 'Groq': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceLLMModel': '', 'groqLLMModel': '', 'azureOAILLMModel': '', 'googleVertexAILLMModel': '', 'ollamaLLMModel': '', 'generateSyntheticData': True, 'optimization': 'bayesianOptimization', 'evalFramework': 'RAGAS', 'evalEmbedding': 'OpenAI:text-embedding-3-large', 'evalLLM': 'OpenAI:gpt-4o-mini', 'sotaEmbeddingModel': 'OpenAI:text-embedding-3-large', 'sotaLLMModel': 'OpenAI:gpt-4o-mini', 'compressors': {'LongContextReorder': True, 'CrossEncoderReranker': True, 'EmbeddingsRedundantFilter': False, 'EmbeddingsClusteringFilter': False, 'LLMChainFilter': False}, 'syntheticDataGeneration': {'testSize': '20', 'criticLLM': 'OpenAI:gpt-4o', 'generatorLLM': 'OpenAI:gpt-4o', 'generatorEmbedding': 'OpenAI:text-embedding-3-large'}, 'testDataPath': None, 'existingSynthDataPath': None, 'testSize': None, 'criticLLM': None, 'generatorLLM': None, 'generatorEmbedding': None, 'numRuns': '50'}
[INFO] 2024-08-26 15:37:41 - ragbuilder.py - Disabled options: ['EmbeddingsRedundantFilter', 'EmbeddingsClusteringFilter', 'LLMChainFilter', 'OpenAI:gpt-4o', 'OpenAI:gpt-3.5-turbo', 'OpenAI:gpt-4-turbo', 'HuggingFace', 'Groq', 'AzureOAI', 'GoogleVertexAI', 'Ollama', 'search_k_10', 'search_k_20', 'OpenAI:text-embedding-3-small', 'OpenAI:text-embedding-ada-002', 'HuggingFace', 'AzureOAI', 'GoogleVertexAI', 'Ollama']
[INFO] 2024-08-26 15:37:41 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:37:41 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:37:41 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:37:41 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:37:41 - embedding.py - getEmbedding Invoked
[INFO] 2024-08-26 15:37:41 - embedding.py - OpenAIEmbeddings Invoked: OpenAI:text-embedding-3-large
[INFO] 2024-08-26 15:37:41 - generate_data.py - Loading docs...
[INFO] 2024-08-26 15:37:41 - loader.py - ragbuilder_loader Invoked:{'input_path': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf_only', 'return_code': False}
[INFO] 2024-08-26 15:37:41 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:37:41 - loader.py - Source type identified: directory
[INFO] 2024-08-26 15:37:41 - loader.py - ragbuilder_directory_loader_exec Invoked
[ERROR] 2024-08-26 15:37:41 - loader.py - Error in ragbuilder_directory_loader_exec: Detected a JSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output.
[ERROR] 2024-08-26 15:37:41 - generate_data.py - Error loading docs for synthetic test data generation.
[ERROR] 2024-08-26 15:37:41 - ragbuilder.py - Synthetic test data generation failed.
[INFO] 2024-08-26 15:37:41 - common.py - INFO: 127.0.0.1:64266 - "POST /rbuilder HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:39:22 - common.py - INFO: 127.0.0.1:64320 - "GET / HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:39:22 - common.py - INFO: 127.0.0.1:64320 - "GET /static/main.js HTTP/1.1" 304 Not Modified
[INFO] 2024-08-26 15:39:22 - common.py - INFO: 127.0.0.1:64320 - "GET /static/ragbuilder_light.png HTTP/1.1" 304 Not Modified
[INFO] 2024-08-26 15:39:22 - common.py - INFO: 127.0.0.1:64320 - "GET /favicon.ico HTTP/1.1" 404 Not Found
[INFO] 2024-08-26 15:39:24 - common.py - INFO: 127.0.0.1:64320 - "GET / HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:39:27 - common.py - INFO: 127.0.0.1:64320 - "GET / HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:39:38 - common.py - INFO: 127.0.0.1:64321 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:39:52 - common.py - INFO: 127.0.0.1:64328 - "POST /check_source_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:40:21 - common.py - INFO: 127.0.0.1:64333 - "GET /templates HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:40:21 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:40:21 - common.py - INFO: 127.0.0.1:64332 - "POST /check_test_data HTTP/1.1" 200 OK
[INFO] 2024-08-26 15:40:36 - ragbuilder.py - enable_analytics = True
[INFO] 2024-08-26 15:40:36 - ragbuilder.py - Initiating parsing config: {'description': 'WFP QA2', 'sourceData': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf_only', 'compareTemplates': True, 'includeNonTemplated': True, 'selectedTemplates': ['query_rewrite', 'step_back_prompt', 'semantic_chunker', 'hyde', 'hybrid_rag', 'rrf'], 'chunkingStrategy': {'MarkdownHeaderTextSplitter': True, 'HTMLHeaderTextSplitter': True, 'SemanticChunker': True, 'RecursiveCharacterTextSplitter': True, 'CharacterTextSplitter': True}, 'chunkSize': {'min': 500, 'max': 2000}, 'embeddingModel': {'OpenAI:text-embedding-3-small': False, 'OpenAI:text-embedding-3-large': True, 'OpenAI:text-embedding-ada-002': False, 'HuggingFace': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceEmbeddingModel': '', 'azureOAIEmbeddingModel': '', 'googleVertexAIEmbeddingModel': '', 'ollamaEmbeddingModel': '', 'vectorDB': 'chromaDB', 'retriever': {'vectorSimilarity': True, 'vectorMMR': True, 'bm25Retriever': True, 'multiQuery': True, 'parentDocFullDoc': True, 'parentDocLargeChunk': True}, 'topK': {'search_k_5': True, 'search_k_10': False, 'search_k_20': False}, 'contextualCompression': True, 'llm': {'OpenAI:gpt-4o-mini': True, 'OpenAI:gpt-4o': False, 'OpenAI:gpt-3.5-turbo': False, 'OpenAI:gpt-4-turbo': False, 'HuggingFace': False, 'Groq': False, 'AzureOAI': False, 'GoogleVertexAI': False, 'Ollama': False}, 'huggingfaceLLMModel': '', 'groqLLMModel': '', 'azureOAILLMModel': '', 'googleVertexAILLMModel': '', 'ollamaLLMModel': '', 'generateSyntheticData': True, 'optimization': 'bayesianOptimization', 'evalFramework': 'RAGAS', 'evalEmbedding': 'OpenAI:text-embedding-3-large', 'evalLLM': 'OpenAI:gpt-4o-mini', 'sotaEmbeddingModel': 'OpenAI:text-embedding-3-large', 'sotaLLMModel': 'OpenAI:gpt-4o-mini', 'compressors': {'LongContextReorder': True, 'CrossEncoderReranker': True, 'EmbeddingsRedundantFilter': False, 'EmbeddingsClusteringFilter': False, 'LLMChainFilter': False}, 'syntheticDataGeneration': {'testSize': '20', 'criticLLM': 'OpenAI:gpt-4o', 'generatorLLM': 'OpenAI:gpt-4o', 'generatorEmbedding': 'OpenAI:text-embedding-3-large'}, 'testDataPath': None, 'existingSynthDataPath': None, 'testSize': None, 'criticLLM': None, 'generatorLLM': None, 'generatorEmbedding': None, 'numRuns': '50'}
[INFO] 2024-08-26 15:40:36 - ragbuilder.py - Disabled options: ['EmbeddingsRedundantFilter', 'EmbeddingsClusteringFilter', 'LLMChainFilter', 'OpenAI:gpt-4o', 'OpenAI:gpt-3.5-turbo', 'OpenAI:gpt-4-turbo', 'HuggingFace', 'Groq', 'AzureOAI', 'GoogleVertexAI', 'Ollama', 'search_k_10', 'search_k_20', 'OpenAI:text-embedding-3-small', 'OpenAI:text-embedding-ada-002', 'HuggingFace', 'AzureOAI', 'GoogleVertexAI', 'Ollama']
[INFO] 2024-08-26 15:40:36 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:40:36 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:40:36 - llmConfig.py - LLM Invoked
[INFO] 2024-08-26 15:40:36 - llmConfig.py - LLM Code Gen Invoked: OpenAI:gpt-4o
[INFO] 2024-08-26 15:40:36 - embedding.py - getEmbedding Invoked
[INFO] 2024-08-26 15:40:36 - embedding.py - OpenAIEmbeddings Invoked: OpenAI:text-embedding-3-large
[INFO] 2024-08-26 15:40:36 - generate_data.py - Loading docs...
[INFO] 2024-08-26 15:40:36 - loader.py - ragbuilder_loader Invoked:{'input_path': '/Users/matthewharris/Desktop/git/llm_eval_research/data/pdf_only', 'return_code': False}
[INFO] 2024-08-26 15:40:36 - loader.py - classify_path Invoked
[INFO] 2024-08-26 15:40:36 - loader.py - Source type identified: directory
[INFO] 2024-08-26 15:40:36 - loader.py - ragbuilder_directory_loader_exec Invoked
[INFO] 2024-08-26 15:42:24 - generate_data.py - Completed loading docs
[INFO] 2024-08-26 15:42:26 - generate_data.py - Initiating synthetic data generation...
[INFO] 2024-08-26 15:42:26 - common.py -
embedding nodes: 0%| | 0/2236 [00:00<?, ?it/s]
[INFO] 2024-08-26 15:42:29 - common.py -
embedding nodes: 0%| | 1/2236 [00:02<1:29:31, 2.40s/it]
[INFO] 2024-08-26 15:42:29 - common.py -
embedding nodes: 0%| | 4/2236 [00:02<18:29, 2.01it/s]
[INFO] 2024-08-26 15:42:29 - common.py -
embedding nodes: 0%| | 9/2236 [00:02<07:57, 4.66it/s]
[INFO] 2024-08-26 15:42:29 - common.py -
embedding nodes: 1%| | 14/2236 [00:03<04:30, 8.23it/s]
[INFO] 2024-08-26 15:42:30 - common.py -
embedding nodes: 1%| | 17/2236 [00:03<03:51, 9.57it/s]
[INFO] 2024-08-26 15:42:30 - common.py -
embedding nodes: 1%| | 21/2236 [00:03<02:52, 12.83it/s]
[INFO] 2024-08-26 15:42:30 - common.py -
embedding nodes: 1%|1 | 24/2236 [00:03<03:21, 10.95it/s]
[INFO] 2024-08-26 15:42:30 - common.py -
embedding nodes: 1%|1 | 28/2236 [00:04<03:13, 11.39it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 1%|1 | 33/2236 [00:04<02:28, 14.80it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 2%|1 | 36/2236 [00:04<02:10, 16.84it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 2%|1 | 39/2236 [00:04<01:56, 18.92it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 2%|1 | 42/2236 [00:04<01:52, 19.55it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 2%|2 | 45/2236 [00:04<02:00, 18.12it/s]
[INFO] 2024-08-26 15:42:31 - common.py -
embedding nodes: 2%|2 | 49/2236 [00:04<01:48, 20.15it/s]
[INFO] 2024-08-26 15:42:32 - common.py -
embedding nodes: 2%|2 | 52/2236 [00:05<01:57, 18.58it/s]
[INFO] 2024-08-26 15:42:32 - common.py -
embedding nodes: 2%|2 | 55/2236 [00:05<02:40, 13.58it/s]
[INFO] 2024-08-26 15:42:32 - common.py -
embedding nodes: 3%|2 | 57/2236 [00:05<02:31, 14.40it/s]
[INFO] 2024-08-26 15:42:32 - common.py -
embedding nodes: 3%|2 | 60/2236 [00:05<02:11, 16.51it/s]
[INFO] 2024-08-26 15:42:33 - common.py -
embedding nodes: 3%|2 | 63/2236 [00:06<03:00, 12.03it/s]
[INFO] 2024-08-26 15:42:33 - common.py -
embedding nodes: 3%|2 | 65/2236 [00:06<02:58, 12.15it/s]
[INFO] 2024-08-26 15:42:33 - common.py -
embedding nodes: 3%|2 | 67/2236 [00:06<03:38, 9.94it/s]
[INFO] 2024-08-26 15:42:33 - common.py -
embedding nodes: 3%|3 | 69/2236 [00:07<04:47, 7.54it/s]
[INFO] 2024-08-26 15:42:34 - common.py -
embedding nodes: 3%|3 | 70/2236 [00:07<06:03, 5.96it/s]
[INFO] 2024-08-26 15:42:34 - common.py -
embedding nodes: 3%|3 | 72/2236 [00:07<06:38, 5.43it/s]
[INFO] 2024-08-26 15:42:35 - common.py -
embedding nodes: 3%|3 | 73/2236 [00:08<07:56, 4.54it/s]
[INFO] 2024-08-26 15:42:35 - common.py -
embedding nodes: 3%|3 | 74/2236 [00:08<10:51, 3.32it/s]
[INFO] 2024-08-26 15:42:36 - common.py -
embedding nodes: 3%|3 | 75/2236 [00:09<16:18, 2.21it/s]
[INFO] 2024-08-26 15:42:37 - common.py -
embedding nodes: 3%|3 | 76/2236 [00:10<21:08, 1.70it/s]
[INFO] 2024-08-26 15:42:38 - common.py -
embedding nodes: 3%|3 | 77/2236 [00:11<24:48, 1.45it/s]
[INFO] 2024-08-26 15:42:39 - common.py -
embedding nodes: 3%|3 | 78/2236 [00:12<22:07, 1.63it/s]
[INFO] 2024-08-26 15:42:39 - common.py -
embedding nodes: 4%|3 | 79/2236 [00:12<20:47, 1.73it/s]
[INFO] 2024-08-26 15:42:40 - common.py -
embedding nodes: 4%|3 | 80/2236 [00:13<18:56, 1.90it/s]
[INFO] 2024-08-26 15:42:40 - common.py -
embedding nodes: 4%|3 | 81/2236 [00:13<17:48, 2.02it/s]
[INFO] 2024-08-26 15:42:41 - common.py -
embedding nodes: 4%|3 | 83/2236 [00:14<20:56, 1.71it/s]
[INFO] 2024-08-26 15:42:44 - common.py -
embedding nodes: 4%|3 | 84/2236 [00:17<34:59, 1.02it/s]
[INFO] 2024-08-26 15:42:45 - common.py -
embedding nodes: 4%|3 | 85/2236 [00:18<41:35, 1.16s/it]
[INFO] 2024-08-26 15:42:46 - common.py -
embedding nodes: 4%|3 | 86/2236 [00:19<34:58, 1.02it/s]
[INFO] 2024-08-26 15:42:46 - common.py -
embedding nodes: 4%|3 | 87/2236 [00:19<29:11, 1.23it/s]
[INFO] 2024-08-26 15:42:47 - common.py -
embedding nodes: 4%|3 | 88/2236 [00:20<25:11, 1.42it/s]
[INFO] 2024-08-26 15:42:47 - common.py -
embedding nodes: 4%|3 | 89/2236 [00:20<21:20, 1.68it/s]
[INFO] 2024-08-26 15:42:47 - common.py -
embedding nodes: 4%|4 | 90/2236 [00:20<16:09, 2.21it/s]
[INFO] 2024-08-26 15:42:49 - common.py -
embedding nodes: 4%|4 | 91/2236 [00:22<29:52, 1.20it/s]
[INFO] 2024-08-26 15:42:52 - common.py -
embedding nodes: 4%|4 | 92/2236 [00:25<52:53, 1.48s/it]
[INFO] 2024-08-26 15:42:53 - common.py -
embedding nodes: 4%|4 | 93/2236 [00:26<51:47, 1.45s/it]
[INFO] 2024-08-26 15:42:54 - common.py -
embedding nodes: 4%|4 | 94/2236 [00:27<41:15, 1.16s/it]
[INFO] 2024-08-26 15:42:54 - common.py -
embedding nodes: 4%|4 | 95/2236 [00:27<32:43, 1.09it/s]
[INFO] 2024-08-26 15:42:54 - common.py -
embedding nodes: 4%|4 | 96/2236 [00:27<26:09, 1.36it/s]
[INFO] 2024-08-26 15:42:55 - common.py -
embedding nodes: 4%|4 | 98/2236 [00:28<19:32, 1.82it/s]
[INFO] 2024-08-26 15:42:57 - common.py -
embedding nodes: 4%|4 | 99/2236 [00:30<29:29, 1.21it/s]
[INFO] 2024-08-26 15:42:59 - common.py -
embedding nodes: 4%|4 | 100/2236 [00:32<44:11, 1.24s/it]
[INFO] 2024-08-26 15:43:00 - common.py -
embedding nodes: 5%|4 | 101/2236 [00:33<43:49, 1.23s/it]
[INFO] 2024-08-26 15:43:01 - common.py -
embedding nodes: 5%|4 | 102/2236 [00:34<36:51, 1.04s/it]
[INFO] 2024-08-26 15:43:01 - common.py -
embedding nodes: 5%|4 | 103/2236 [00:34<31:47, 1.12it/s]
[INFO] 2024-08-26 15:43:02 - common.py -
embedding nodes: 5%|4 | 104/2236 [00:35<26:05, 1.36it/s]
[INFO] 2024-08-26 15:43:02 - common.py -
embedding nodes: 5%|4 | 105/2236 [00:35<22:47, 1.56it/s]
[INFO] 2024-08-26 15:43:06 - common.py -
embedding nodes: 5%|4 | 107/2236 [00:39<48:13, 1.36s/it]
[INFO] 2024-08-26 15:43:08 - common.py -
embedding nodes: 5%|4 | 108/2236 [00:41<49:19, 1.39s/it]
[INFO] 2024-08-26 15:43:08 - common.py -
embedding nodes: 5%|4 | 109/2236 [00:41<39:56, 1.13s/it]
[INFO] 2024-08-26 15:43:09 - common.py -
embedding nodes: 5%|4 | 110/2236 [00:42<34:28, 1.03it/s]
[INFO] 2024-08-26 15:43:09 - common.py -
embedding nodes: 5%|4 | 111/2236 [00:42<30:26, 1.16it/s]
[INFO] 2024-08-26 15:43:10 - common.py -
embedding nodes: 5%|5 | 112/2236 [00:43<25:42, 1.38it/s]
[INFO] 2024-08-26 15:43:10 - common.py -
embedding nodes: 5%|5 | 113/2236 [00:43<21:35, 1.64it/s]
[INFO] 2024-08-26 15:43:10 - common.py -
embedding nodes: 5%|5 | 114/2236 [00:43<16:39, 2.12it/s]
[INFO] 2024-08-26 15:43:11 - common.py -
embedding nodes: 5%|5 | 115/2236 [00:44<23:53, 1.48it/s]
[INFO] 2024-08-26 15:43:15 - common.py -
embedding nodes: 5%|5 | 116/2236 [00:48<49:43, 1.41s/it]
[INFO] 2024-08-26 15:43:16 - common.py -
embedding nodes: 5%|5 | 117/2236 [00:49<46:30, 1.32s/it]
[INFO] 2024-08-26 15:43:16 - common.py -
embedding nodes: 5%|5 | 118/2236 [00:49<38:07, 1.08s/it]
[INFO] 2024-08-26 15:43:17 - common.py -
embedding nodes: 5%|5 | 119/2236 [00:50<30:31, 1.16it/s]
[INFO] 2024-08-26 15:43:17 - common.py -
embedding nodes: 5%|5 | 120/2236 [00:50<25:38, 1.37it/s]
[INFO] 2024-08-26 15:43:17 - common.py -
embedding nodes: 5%|5 | 121/2236 [00:50<20:06, 1.75it/s]
[INFO] 2024-08-26 15:43:17 - common.py -
embedding nodes: 5%|5 | 122/2236 [00:50<16:25, 2.14it/s]
[INFO] 2024-08-26 15:43:19 - common.py -
embedding nodes: 6%|5 | 123/2236 [00:52<29:10, 1.21it/s]
[INFO] 2024-08-26 15:43:21 - common.py -
embedding nodes: 6%|5 | 124/2236 [00:54<37:46, 1.07s/it]
[INFO] 2024-08-26 15:43:22 - common.py -
embedding nodes: 6%|5 | 125/2236 [00:55<41:37, 1.18s/it]
[INFO] 2024-08-26 15:43:23 - common.py -
embedding nodes: 6%|5 | 126/2236 [00:56<32:52, 1.07it/s]
[INFO] 2024-08-26 15:43:23 - common.py -
embedding nodes: 6%|5 | 127/2236 [00:56<26:59, 1.30it/s]
[INFO] 2024-08-26 15:43:23 - common.py -
embedding nodes: 6%|5 | 128/2236 [00:56<23:12, 1.51it/s]
[INFO] 2024-08-26 15:43:23 - common.py -
embedding nodes: 6%|5 | 129/2236 [00:56<17:22, 2.02it/s]
[INFO] 2024-08-26 15:43:24 - common.py -
embedding nodes: 6%|5 | 130/2236 [00:57<14:25, 2.43it/s]
[INFO] 2024-08-26 15:43:24 - common.py -
embedding nodes: 6%|5 | 131/2236 [00:57<12:06, 2.90it/s]
[INFO] 2024-08-26 15:43:26 - common.py -
embedding nodes: 6%|5 | 132/2236 [00:59<31:29, 1.11it/s]
[INFO] 2024-08-26 15:43:27 - common.py -
embedding nodes: 6%|5 | 133/2236 [01:00<35:43, 1.02s/it]
[INFO] 2024-08-26 15:43:28 - common.py -
embedding nodes: 6%|5 | 134/2236 [01:01<28:39, 1.22it/s]
[INFO] 2024-08-26 15:43:28 - common.py -
embedding nodes: 6%|6 | 135/2236 [01:01<23:40, 1.48it/s]
[INFO] 2024-08-26 15:43:28 - common.py -
embedding nodes: 6%|6 | 136/2236 [01:01<20:17, 1.72it/s]
[INFO] 2024-08-26 15:43:29 - common.py -
embedding nodes: 6%|6 | 137/2236 [01:02<18:09, 1.93it/s]
[INFO] 2024-08-26 15:43:31 - common.py -
embedding nodes: 6%|6 | 139/2236 [01:04<24:36, 1.42it/s]
[INFO] 2024-08-26 15:43:33 - common.py -
embedding nodes: 6%|6 | 140/2236 [01:06<37:55, 1.09s/it]
[INFO] 2024-08-26 15:43:34 - common.py -
embedding nodes: 6%|6 | 141/2236 [01:07<38:57, 1.12s/it]
[INFO] 2024-08-26 15:43:34 - common.py -
embedding nodes: 6%|6 | 142/2236 [01:08<32:44, 1.07it/s]
[INFO] 2024-08-26 15:43:35 - common.py -
embedding nodes: 6%|6 | 143/2236 [01:08<27:22, 1.27it/s]
[INFO] 2024-08-26 15:43:35 - common.py -
embedding nodes: 6%|6 | 144/2236 [01:08<23:23, 1.49it/s]
[INFO] 2024-08-26 15:43:36 - common.py -
embedding nodes: 6%|6 | 145/2236 [01:09<19:25, 1.79it/s]
[INFO] 2024-08-26 15:43:36 - common.py -
embedding nodes: 7%|6 | 146/2236 [01:09<16:48, 2.07it/s]
[INFO] 2024-08-26 15:43:37 - common.py -
embedding nodes: 7%|6 | 147/2236 [01:10<28:25, 1.22it/s]
[INFO] 2024-08-26 15:43:39 - common.py -
embedding nodes: 7%|6 | 148/2236 [01:12<39:39, 1.14s/it]
[INFO] 2024-08-26 15:43:41 - common.py -
embedding nodes: 7%|6 | 149/2236 [01:14<40:11, 1.16s/it]
[INFO] 2024-08-26 15:43:41 - common.py -
embedding nodes: 7%|6 | 150/2236 [01:14<34:36, 1.00it/s]
[INFO] 2024-08-26 15:43:42 - common.py -
embedding nodes: 7%|6 | 151/2236 [01:15<28:39, 1.21it/s]
[INFO] 2024-08-26 15:43:42 - common.py -
embedding nodes: 7%|6 | 152/2236 [01:15<24:21, 1.43it/s]
[INFO] 2024-08-26 15:43:42 - common.py -
embedding nodes: 7%|6 | 153/2236 [01:15<20:31, 1.69it/s]
[INFO] 2024-08-26 15:43:43 - common.py -
embedding nodes: 7%|6 | 154/2236 [01:16<20:16, 1.71it/s]
[INFO] 2024-08-26 15:43:44 - common.py -
embedding nodes: 7%|6 | 155/2236 [01:18<30:35, 1.13it/s]
[INFO] 2024-08-26 15:43:47 - common.py -
embedding nodes: 7%|6 | 156/2236 [01:20<42:35, 1.23s/it]
[INFO] 2024-08-26 15:43:48 - common.py -
embedding nodes: 7%|7 | 157/2236 [01:21<39:59, 1.15s/it]
[INFO] 2024-08-26 15:43:48 - common.py -
embedding nodes: 7%|7 | 158/2236 [01:21<34:42, 1.00s/it]
[INFO] 2024-08-26 15:43:49 - common.py -
embedding nodes: 7%|7 | 159/2236 [01:22<30:02, 1.15it/s]
[INFO] 2024-08-26 15:43:49 - common.py -
embedding nodes: 7%|7 | 160/2236 [01:22<24:41, 1.40it/s]
[INFO] 2024-08-26 15:43:49 - common.py -
embedding nodes: 7%|7 | 161/2236 [01:22<18:23, 1.88it/s]
[INFO] 2024-08-26 15:43:49 - common.py -
embedding nodes: 7%|7 | 162/2236 [01:22<15:46, 2.19it/s]
[INFO] 2024-08-26 15:43:51 - common.py -
embedding nodes: 7%|7 | 163/2236 [01:24<27:49, 1.24it/s]
[INFO] 2024-08-26 15:43:53 - common.py -
embedding nodes: 7%|7 | 164/2236 [01:26<39:11, 1.13s/it]
[INFO] 2024-08-26 15:43:55 - common.py -
embedding nodes: 7%|7 | 165/2236 [01:28<45:43, 1.32s/it]
[INFO] 2024-08-26 15:43:55 - common.py -
embedding nodes: 7%|7 | 166/2236 [01:28<36:54, 1.07s/it]
[INFO] 2024-08-26 15:43:56 - common.py -
embedding nodes: 7%|7 | 167/2236 [01:29<29:44, 1.16it/s]
[INFO] 2024-08-26 15:43:56 - common.py -
embedding nodes: 8%|7 | 168/2236 [01:29<26:25, 1.30it/s]
[INFO] 2024-08-26 15:43:57 - common.py -
embedding nodes: 8%|7 | 169/2236 [01:30<22:43, 1.52it/s]
[INFO] 2024-08-26 15:43:57 - common.py -
embedding nodes: 8%|7 | 170/2236 [01:30<18:09, 1.90it/s]
[INFO] 2024-08-26 15:43:58 - common.py -
embedding nodes: 8%|7 | 171/2236 [01:31<28:59, 1.19it/s]
[INFO] 2024-08-26 15:44:00 - common.py -
embedding nodes: 8%|7 | 172/2236 [01:33<41:38, 1.21s/it]
[INFO] 2024-08-26 15:44:02 - common.py -
embedding nodes: 8%|7 | 173/2236 [01:35<42:48, 1.24s/it]
[INFO] 2024-08-26 15:44:02 - common.py -
embedding nodes: 8%|7 | 174/2236 [01:35<34:39, 1.01s/it]
[INFO] 2024-08-26 15:44:03 - common.py -
embedding nodes: 8%|7 | 175/2236 [01:36<27:52, 1.23it/s]
[INFO] 2024-08-26 15:44:03 - common.py -
embedding nodes: 8%|7 | 176/2236 [01:36<23:39, 1.45it/s]
[INFO] 2024-08-26 15:44:03 - common.py -
embedding nodes: 8%|7 | 177/2236 [01:36<21:05, 1.63it/s]
[INFO] 2024-08-26 15:44:04 - common.py -
embedding nodes: 8%|7 | 178/2236 [01:37<19:30, 1.76it/s]
[INFO] 2024-08-26 15:44:04 - common.py -
embedding nodes: 8%|8 | 179/2236 [01:37<19:32, 1.75it/s]
[INFO] 2024-08-26 15:44:06 - common.py -
embedding nodes: 8%|8 | 180/2236 [01:39<33:20, 1.03it/s]
[INFO] 2024-08-26 15:44:08 - common.py -
embedding nodes: 8%|8 | 181/2236 [01:41<40:48, 1.19s/it]
[INFO] 2024-08-26 15:44:10 - common.py -
embedding nodes: 8%|8 | 182/2236 [01:43<50:42, 1.48s/it]
[INFO] 2024-08-26 15:44:11 - common.py -
embedding nodes: 8%|8 | 183/2236 [01:44<40:07, 1.17s/it]
[INFO] 2024-08-26 15:44:11 - common.py -
embedding nodes: 8%|8 | 184/2236 [01:44<31:21, 1.09it/s]
[INFO] 2024-08-26 15:44:11 - common.py -
embedding nodes: 8%|8 | 185/2236 [01:44<25:48, 1.32it/s]
[INFO] 2024-08-26 15:44:12 - common.py -
embedding nodes: 8%|8 | 186/2236 [01:45<21:56, 1.56it/s]
[INFO] 2024-08-26 15:44:14 - common.py -
embedding nodes: 8%|8 | 188/2236 [01:47<32:13, 1.06it/s]
[INFO] 2024-08-26 15:44:17 - common.py -
embedding nodes: 8%|8 | 189/2236 [01:50<49:07, 1.44s/it]
[INFO] 2024-08-26 15:44:18 - common.py -
embedding nodes: 8%|8 | 190/2236 [01:51<44:10, 1.30s/it]
[INFO] 2024-08-26 15:44:19 - common.py -
embedding nodes: 9%|8 | 191/2236 [01:52<42:55, 1.26s/it]
[INFO] 2024-08-26 15:44:20 - common.py -
embedding nodes: 9%|8 | 192/2236 [01:53<34:09, 1.00s/it]
[INFO] 2024-08-26 15:44:20 - common.py -
embedding nodes: 9%|8 | 193/2236 [01:53<29:20, 1.16it/s]
[INFO] 2024-08-26 15:44:21 - common.py -
embedding nodes: 9%|8 | 194/2236 [01:54<24:27, 1.39it/s]
[INFO] 2024-08-26 15:44:21 - common.py -
embedding nodes: 9%|8 | 195/2236 [01:54<18:18, 1.86it/s]
[INFO] 2024-08-26 15:44:22 - common.py -
embedding nodes: 9%|8 | 196/2236 [01:55<30:51, 1.10it/s]
[INFO] 2024-08-26 15:44:24 - common.py -
embedding nodes: 9%|8 | 197/2236 [01:57<39:56, 1.18s/it]
[INFO] 2024-08-26 15:44:26 - common.py -
embedding nodes: 9%|8 | 198/2236 [01:59<48:35, 1.43s/it]
[INFO] 2024-08-26 15:44:27 - common.py -
embedding nodes: 9%|8 | 199/2236 [02:00<38:53, 1.15s/it]
[INFO] 2024-08-26 15:44:27 - common.py -
embedding nodes: 9%|8 | 200/2236 [02:00<31:39, 1.07it/s]
[INFO] 2024-08-26 15:44:28 - common.py -
embedding nodes: 9%|8 | 201/2236 [02:01<25:53, 1.31it/s]
[INFO] 2024-08-26 15:44:28 - common.py -
embedding nodes: 9%|9 | 202/2236 [02:01<21:50, 1.55it/s]
[INFO] 2024-08-26 15:44:28 - common.py -
embedding nodes: 9%|9 | 203/2236 [02:01<19:18, 1.75it/s]
[INFO] 2024-08-26 15:44:29 - common.py -
embedding nodes: 9%|9 | 204/2236 [02:02<21:14, 1.59it/s]
[INFO] 2024-08-26 15:44:31 - common.py -
embedding nodes: 9%|9 | 205/2236 [02:04<36:21, 1.07s/it]
[INFO] 2024-08-26 15:44:33 - common.py -
embedding nodes: 9%|9 | 206/2236 [02:06<41:58, 1.24s/it]
[INFO] 2024-08-26 15:44:35 - common.py -
embedding nodes: 9%|9 | 207/2236 [02:08<53:24, 1.58s/it]
[INFO] 2024-08-26 15:44:37 - common.py -
embedding nodes: 9%|9 | 208/2236 [02:10<51:57, 1.54s/it]
[INFO] 2024-08-26 15:44:37 - common.py -
embedding nodes: 9%|9 | 209/2236 [02:10<41:30, 1.23s/it]
[INFO] 2024-08-26 15:44:38 - common.py -
embedding nodes: 9%|9 | 210/2236 [02:11<33:49, 1.00s/it]
[INFO] 2024-08-26 15:44:38 - common.py -
embedding nodes: 9%|9 | 211/2236 [02:11<27:11, 1.24it/s]
[INFO] 2024-08-26 15:44:38 - common.py -
embedding nodes: 9%|9 | 212/2236 [02:11<22:49, 1.48it/s]
[INFO] 2024-08-26 15:44:39 - common.py -
embedding nodes: 10%|9 | 213/2236 [02:12<19:51, 1.70it/s]
[INFO] 2024-08-26 15:44:40 - common.py -
embedding nodes: 10%|9 | 214/2236 [02:13<29:11, 1.15it/s]
[INFO] 2024-08-26 15:44:42 - common.py -
embedding nodes: 10%|9 | 215/2236 [02:15<41:11, 1.22s/it]
[INFO] 2024-08-26 15:44:45 - common.py -
embedding nodes: 10%|9 | 216/2236 [02:18<54:54, 1.63s/it]
[INFO] 2024-08-26 15:44:46 - common.py -
Exception in thread Thread-2:
[INFO] 2024-08-26 15:44:46 - common.py - Traceback (most recent call last):
[INFO] 2024-08-26 15:44:46 - common.py - File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
[INFO] 2024-08-26 15:44:46 - common.py - self.run()
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/executor.py", line 96, in run
[INFO] 2024-08-26 15:44:46 - common.py - results = self.loop.run_until_complete(self._aresults())
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/executor.py", line 84, in _aresults
[INFO] 2024-08-26 15:44:46 - common.py - raise e
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/executor.py", line 79, in _aresults
[INFO] 2024-08-26 15:44:46 - common.py - r = await future
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 605, in _wait_for_one
[INFO] 2024-08-26 15:44:46 - common.py - return f.result() # May raise f.exception().
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/executor.py", line 38, in sema_coro
[INFO] 2024-08-26 15:44:46 - common.py - return await coro
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/executor.py", line 112, in wrapped_callable_async
[INFO] 2024-08-26 15:44:46 - common.py - return counter, await callable(*args, **kwargs)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/testset/extractor.py", line 49, in extract
[INFO] 2024-08-26 15:44:46 - common.py - results = await self.llm.generate(prompt=prompt, is_async=is_async)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/llms/base.py", line 92, in generate
[INFO] 2024-08-26 15:44:46 - common.py - return await agenerate_text_with_retry(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/asyncio/init.py", line 189, in async_wrapped
[INFO] 2024-08-26 15:44:46 - common.py - return await copy(fn, *args, **kwargs)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/asyncio/init.py", line 111, in call
[INFO] 2024-08-26 15:44:46 - common.py - do = await self.iter(retry_state=retry_state)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/asyncio/init.py", line 153, in iter
[INFO] 2024-08-26 15:44:46 - common.py - result = await action(retry_state)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
[INFO] 2024-08-26 15:44:46 - common.py - return call(*args, **kwargs)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/init.py", line 418, in exc_check
[INFO] 2024-08-26 15:44:46 - common.py - raise retry_exc.reraise()
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/init.py", line 185, in reraise
[INFO] 2024-08-26 15:44:46 - common.py - raise self.last_attempt.result()
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 449, in result
[INFO] 2024-08-26 15:44:46 - common.py - return self.__get_result()
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
[INFO] 2024-08-26 15:44:46 - common.py - raise self._exception
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/tenacity/asyncio/init.py", line 114, in call
[INFO] 2024-08-26 15:44:46 - common.py - result = await fn(*args, **kwargs)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/ragas/llms/base.py", line 169, in agenerate_text
[INFO] 2024-08-26 15:44:46 - common.py - return await self.langchain_llm.agenerate_prompt(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 708, in agenerate_prompt
[INFO] 2024-08-26 15:44:46 - common.py - return await self.agenerate(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 668, in agenerate
[INFO] 2024-08-26 15:44:46 - common.py - raise exceptions[0]
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 853, in _agenerate_with_cache
[INFO] 2024-08-26 15:44:46 - common.py - result = await self._agenerate(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/langchain_openai/chat_models/base.py", line 740, in _agenerate
[INFO] 2024-08-26 15:44:46 - common.py - response = await self.async_client.create(**payload)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1339, in create
[INFO] 2024-08-26 15:44:46 - common.py - return await self._post(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
[INFO] 2024-08-26 15:44:46 - common.py - return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
[INFO] 2024-08-26 15:44:46 - common.py - return await self._request(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1596, in _request
[INFO] 2024-08-26 15:44:46 - common.py - return await self._retry_request(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1643, in _retry_request
[INFO] 2024-08-26 15:44:46 - common.py - return await self._request(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1596, in _request
[INFO] 2024-08-26 15:44:46 - common.py - return await self._retry_request(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1643, in _retry_request
[INFO] 2024-08-26 15:44:46 - common.py - return await self._request(
[INFO] 2024-08-26 15:44:46 - common.py - ^^^^^^^^^^^^^^^^^^^^
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-doclaynet/env_wfp_rag/lib/python3.11/site-packages/openai/_base_client.py", line 1611, in _request
[INFO] 2024-08-26 15:44:46 - common.py - raise self._make_status_error_from_response(err.response) from None
[INFO] 2024-08-26 15:44:46 - common.py - openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-14onX0p3Xh0qpWBmqQ5WcBUO on tokens per min (TPM): Limit 30000, Used 29613, Requested 1266. Please try again in 1.758s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
[ERROR] 2024-08-26 15:44:46 - ragbuilder.py - Synthetic test data generation failed: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass raise_exceptions=False incase you want to show only a warning message instead.
[INFO] 2024-08-26 15:44:46 - common.py - INFO: 127.0.0.1:64334 - "POST /rbuilder HTTP/1.1" 500 Internal Server Error
[INFO] 2024-08-26 15:44:46 - common.py - ERROR: Exception in ASGI application
[INFO] 2024-08-26 15:44:46 - common.py - Traceback (most recent call last):
[INFO] 2024-08-26 15:44:46 - common.py - File "/Users/matthewharris/Desktop/git/yolo-
... etc
Hi @dividor , sorry for the delay in responding, but we are aware of this issue. Will try to apply some exponential backoff & retry logic to handle this better.
Thanks!
On Mon, 2 Sept 2024 at 13:54, aravind10x @.***> wrote:
Hi @dividor https://github.com/dividor , sorry for the delay in responding, but we are aware of this issue. Will try to apply some exponential backoff & retry logic to handle this better.
— Reply to this email directly, view it on GitHub https://github.com/KruxAI/ragbuilder/issues/28#issuecomment-2325153879, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADNGXAJ4BUZOONBFHGCO3ZUSQ7DAVCNFSM6AAAAABNEWGXRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRVGE2TGOBXHE . You are receiving this because you were mentioned.Message ID: @.***>
Hi @roybarber @dividor, we will have a fix for the rate limit errors this week. Sorry for the delay in prioritizing this!