topoteretes
Embedding issue - Exclude/ignore certain paths in codegraph pipeline
From Discord (Thread ID: 1350021363838222408)
Hello, I'm playing with cognee locally, but I must be doing something wrong. I'm following https://docs.cognee.ai/tutorials/turn_your_repo_into_graph which just loops trying to use the openAI embed models with error messages like: ERROR:LiteLLMEmbeddingEngine:Embedding error with model openai/text-embedding-3-large: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': 'Requested 1086442 tokens, max 600000 tokens per request', 'type': 'max_tokens_per_request', 'param': None, 'code': 'max_tokens_per_request'}}ERROR:cognee.exceptions.exceptions:EmbeddingException: Failed to index data points using model openai/text-embedding-3-large (Status code: 422)WARNING:index_data_points:Failed to index data points for ClassDefinition.source_code: ('Failed to index data points using model openai/text-embedding-3-large' Do I need to tune the EMBEDDING_MAX_TOKENS variable to make things work?
In specific the user wanted the ability to exclude/ignore certain paths like the .venv folder
@dexters1 what is the status of this one? can we close or is there a ticket
