microsoft
[running error] how to use openai ?
Describe the issue
我在利用密钥和代理使用openai官方模型的过程中出错了
Steps to reproduce
使用代理proxy访问openai官方模型，使用个人密钥，其余设置保持默认
GraphRAG Config Used
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: sk-ySJkHe4C。。。。
type: openai_chat # or azure_openai_chat
model: gpt-4-turbo-preview
model_supports_json: true # recommended if this is available for your model.
proxy: http://172.21.51.30:3128
# max_tokens: 4000
# request_timeout: 180.0
# api_base: https://api.172.21.51.30:3128.com
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
# max_retries: 10
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
# concurrent_requests: 25 # the number of parallel inflight requests that may be made
parallelization:
stagger: 0.3
# num_threads: 50 # the number of threads to use for parallel processing
async_mode: threaded # or asyncio
embeddings:
## parallelization: override the global parallelization settings for embeddings
async_mode: threaded # or asyncio
llm:
api_key: sk-ySJkHe4。。。
type: openai_embedding # or azure_openai_embedding
model: text-embedding-3-small
proxy: http://172.21.51.30:3128
# api_base: https://<instance>.openai.azure.com
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
# max_retries: 10
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
# concurrent_requests: 25 # the number of parallel inflight requests that may be made
# batch_size: 16 # the number of documents to send in a single request
# batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
# target: required # or optional
chunks:
size: 300
overlap: 100
group_by_columns: [id] # by default, we don't allow chunks to cross documents
input:
type: file # or blob
file_type: text # or csv
base_dir: "input"
file_encoding: utf-8
file_pattern: ".*\\.txt$"
cache:
type: file # or blob
base_dir: "cache"
# connection_string: <azure_blob_storage_connection_string>
# container_name: <azure_blob_storage_container_name>
Logs and screenshots
graphrag.config.read_dotenv INFO Loading pipeline .env file
17:33:47,38 graphrag.index.cli INFO using default configuration: {
"llm": {
"api_key": "REDACTED, length 51",
"type": "openai_chat",
"model": "gpt-4-turbo-preview",
"max_tokens": 4000,
"request_timeout": 180.0,
"api_base": null,
"api_version": null,
"proxy": null,
"cognitive_services_endpoint": null,
"deployment_name": null,
"model_supports_json": true,
"tokens_per_minute": 0,
"requests_per_minute": 0,
"max_retries": 10,
"max_retry_wait": 10.0,
"sleep_on_rate_limit_recommendation": true,
"concurrent_requests": 25
},
"parallelization": {
"stagger": 0.3,
"num_threads": 50
},
"async_mode": "threaded",
"root_dir": "./ragtest",
"reporting": {
"type": "file",
"base_dir": "output/${timestamp}/reports",
"storage_account_blob_url": null
},
"storage": {
"type": "file",
"base_dir": "output/${timestamp}/artifacts",
"storage_account_blob_url": null
},
"cache": {
"type": "file",
"base_dir": "cache",
"storage_account_blob_url": null
},
"input": {
"type": "file",
"file_type": "text",
"base_dir": "input",
"storage_account_blob_url": null,
"encoding": "utf-8",
"file_pattern": ".*\\.txt$",
"file_filter": null,
"source_column": null,
"timestamp_column": null,
"timestamp_format": null,
"text_column": "text",
"title_column": null,
"document_attribute_columns": []
},
"embed_graph": {
"enabled": false,
"num_walks": 10,
"walk_length": 40,
"window_size": 2,
"iterations": 3,
"random_seed": 597832,
"strategy": null
},
"embeddings": {
"llm": {
"api_key": "REDACTED, length 51",
"type": "openai_embedding",
"model": "text-embedding-3-small",
"max_tokens": 4000,
"request_timeout": 180.0,
"api_base": null,
"api_version": null,
"proxy": "http://172.21.51.30:3128",
"cognitive_services_endpoint": null,
"deployment_name": null,
"model_supports_json": null,
"tokens_per_minute": 0,
"requests_per_minute": 0,
"max_retries": 10,
"max_retry_wait": 10.0,
"sleep_on_rate_limit_recommendation": true,
"concurrent_requests": 25
},
"parallelization": {
"stagger": 0.3,
"num_threads": 50
},
"async_mode": "threaded",
"batch_size": 16,
"batch_max_tokens": 8191,
"target": "required",
"skip": [],
"vector_store": null,
"strategy": null
},
"chunks": {
"size": 300,
"overlap": 100,
"group_by_columns": [
"id"
],
"strategy": null
。。。。
raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
18:26:56,914 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None17:33:47,34
Additional Information
- GraphRAG Version:
- Operating System:linux
- Python Version:3.11
- Related Issues:
This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the tokens_per_minute and requests_per_minute config setting to an appropriate value other than 0. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.
We are looking at better error handling to improve this behavior.
This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the
tokens_per_minuteandrequests_per_minuteconfig setting to an appropriate value other than0. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.We are looking at better error handling to improve this behavior.
@jgbradley1 hi,Why is it that when I change the tokens_per_minute and requests_per_minute values in settings.yaml, but I see that the llm configuration item is not changed to the corresponding value in the log? Only the configuration item in embeddings llm can be modified successfully.
Can you use the proxy http://172.21.51.30:3128 normally? I checked the source code of the latest released version and it seems that the proxy parameter of openaiclient is not passed, and proxy cannot be used under LLM but api_proxy instead. I had to modify it to make it work.
I have a similar problem when running the indexing job
It continuously retries after hitting TMP threshold
The index job is stuck at 6.25% completion
Pipeline configuration : lm: type: azure_openai_chat api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION model: $GRAPHRAG_LLM_MODEL deployment_name: $GRAPHRAG_LLM_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT model_supports_json: True tokens_per_minute: 8000 requests_per_minute: 1 thread_count: 15 concurrent_requests: 1
parallelization: stagger: 0.75 num_threads: 2
async_mode: threaded
embeddings: async_mode: threaded llm: type: azure_openai_embedding api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION batch_size: 16 model: $GRAPHRAG_EMBEDDING_MODEL deployment_name: $GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT tokens_per_minute: 100000 concurrent_requests: 15 requests_per_minute: 1000 thread_count: 25 max_retries: 50 parallelization: stagger: 0.75 num_threads: 10 vector_store: type: azure_ai_search collection_name: PLACEHOLDER title_column: name overwrite: True url: $AI_SEARCH_URL audience: $AI_SEARCH_AUDIENCE
This issue has been closed after being marked as stale for five days. Please reopen if needed.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}