Changed MAX_TOKENS to the correct value of 8190

[ReinforcedKnowledge]

Thank you for contributing to LangChain!

• Changed MAX_TOKENS to the correct value of 8190: "langchain-mistralai"

• Description: Changed the value of MAX_TOKENS, context window size of the MistralAI embed model) in the embeddings

• Issue: #20523

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Apr 25, 2024 0:09am

[ccurme]

Hi @ReinforcedKnowledge, thanks for this. I'm finding that after this change we still get an error for the "hello world" * 4095 case described in the issue, although it is different: {'object': 'error', 'message': 'Please provide at least one input element.', 'type': 'invalid_request_error', 'param': None, 'code': None} Do you find the same?

[ReinforcedKnowledge]

Hi @ccurme, thank you for checking this PR and pointing out the issue. Can you please ensure that you are using the same versions as the one mentioned in the issue Which are:

langchain-core==0.1.43
langchain-mistralai==0.1.2

I'm sure you have verified it but I can't replicate your error message. I still get the exact same error as before. From my understanding of the code, we should get the same error as the one in the issue regardless of that maximum token count because to embed documents we:

• We use this _get_batches generator function to turn a list of strings into an iterable over batches, here, an iterable over a list of strings.
• The way we group the initial documents into batches is by iterating over them and putting them in the same batch as long as their sum of token count doesn't exceed the MAX_TOKENS variable. Once the condition doesn't hold for a document, we yield the batch and put the remaining document into a new batch.

So if we have a document, that intrinsically has a higher token count than MAX_TOKENS, this condition will not hold and we'll send it through the post request and we should receive the same error.

By the way, I'm also adding a small commit to this branch which concerns the docstring of the _get_batches method. Changed the 16k to MAX_TOKENS in the docstring.

[efriis]

Commented on the issue!