handleLLMError has not been called.

[geekroscom]

In what scenarios will handleLLMError be triggered? For example, it did not trigger this callback when I did not set an API KEY or when the API KEY was incorrect.

callbacks: [
    {
        handleLLMStart(llm: any, prompts: any, runId: any, parentRunId: any){
            console.log(llm, prompts, runId, parentRunId);
        },
        handleLLMNewToken(token: any, runId: any, parentRunId: any) {
            console.log(token, runId, parentRunId);
        },
        handleLLMError(err: any, runId: any, parentRunId: any){
            console.log(err, runId, parentRunId);
        },
        handleLLMEnd(outputs: any, runId: any, parentRunId: any){
            console.log(outputs, runId, parentRunId);
        }
    }
]

[geekroscom]

Hello?

[V01D-NULL]

@geekros I am encountering a similar issue where the handleLLMEnd callback is not triggered when using the Azure integration. It might be what you're seeing?

As for the "when" question, llm error events get called when an exception is thrown within the try/catch blocks of either the chat_model base or llm base.

https://github.com/hwchase17/langchainjs/blob/912d21c84d46c1d0c7cda267d0a3e95ee1f10330/langchain/src/chat_models/base.ts#L101

https://github.com/hwchase17/langchainjs/blob/912d21c84d46c1d0c7cda267d0a3e95ee1f10330/langchain/src/llms/base.ts#L114

Edit: Updating from 0.0.72 to the latest version (0.0.92 at the time of writing) fixed it for us

[dosubot[bot]]

Hi, @geekros! I'm Dosu, and I'm here to help the langchainjs team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about the handleLLMError callback not being triggered in certain scenarios. You were seeking clarification on when this callback should be triggered. Another user, V01D-NULL, encountered a similar issue and suggested updating to the latest version of the software, which fixed the problem for them. So it seems that updating to the latest version resolved the issue for them.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the langchainjs repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to langchainjs!

[V01D-NULL]

good bot

[dosubot[bot]]

Thank you, @V01D-NULL, for your response! We appreciate your feedback and are glad to hear that updating to the latest version resolved the issue for you. We will go ahead and close this issue now. If you have any further questions or need assistance in the future, please don't hesitate to reach out.