langchainjs icon indicating copy to clipboard operation
langchainjs copied to clipboard

handleToolEnd not called when using a tool calling agent with ChatAnthropic and Sonnet 3.5

Open zacharyfmarion opened this issue 1 year ago • 3 comments

Checked other resources

  • [X] I added a very descriptive title to this issue.
  • [X] I searched the LangChain.js documentation with the integrated search.
  • [X] I used the GitHub search to find a similar question and didn't find it.
  • [X] I am sure that this is a bug in LangChain.js rather than my code.
  • [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

/* eslint-disable no-console */
import { ChatAnthropic } from '@langchain/anthropic';
import { Calculator } from '@langchain/community/tools/calculator';
import { HumanMessage } from '@langchain/core/messages';
import { ChatPromptTemplate, MessagesPlaceholder } from '@langchain/core/prompts';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';

const model = new ChatAnthropic({
  model: 'claude-3-5-sonnet-20240620',
  temperature: 0,
});
const tools = [new Calculator()];

async function main() {
  const prompt = ChatPromptTemplate.fromMessages([
    ['system', 'You are a math phd student'],
    new MessagesPlaceholder('messages'),
    new MessagesPlaceholder('agent_scratchpad'),
  ]);

  const agent = createToolCallingAgent({
    llm: model,
    tools,
    prompt,
  });

  const executor = new AgentExecutor({
    agent,
    tools,
    callbacks: [
      {
        handleAgentAction() {
          console.log('handleAgentAction');
        },
        handleAgentEnd() {
          console.log('handleAgentEnd');
        },
        handleToolEnd() {
          console.log('handleToolEnd');
        },
      },
    ],
  });

  await executor.invoke({
    messages: [new HumanMessage({ content: 'What is 2 to the power of 4' })],
  });
}

main();

Error Message and Stack Trace (if applicable)

No response

Description

When using createToolCallingAgent with ChatAnthropic and passing in callbacks through an AgentExecutor the handleToolEnd callback does not get executed. Because of this I am not able to track token usage for the created agent.

Current Behavior

When run only two log statements appear:

handleAgentAction
handleAgentEnd

Expected Behavior

I would expect handleToolEnd to also be logged.

System Info

└─ langchain@npm:0.2.8::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Flangchain%2F-%2Flangchain-0.2.8.tgz
   ├─ Instances: 1
   ├─ Version: 0.2.8
   │ 
   └─ Dependencies
      ├─ @langchain/core@npm:>=0.2.9 <0.3.0 → npm:0.2.14::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2F%40langchain%2Fcore%2F-%2Fcore-0.2.14.tgz
      ├─ @langchain/openai@npm:>=0.1.0 <0.3.0 → npm:0.2.1::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2F%40langchain%2Fopenai%2F-%2Fopenai-0.2.1.tgz
      ├─ @langchain/textsplitters@npm:~0.0.0 → npm:0.0.3::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2F%40langchain%2Ftextsplitters%2F-%2Ftextsplitters-0.0.3.tgz
      ├─ binary-extensions@npm:^2.2.0 → npm:2.3.0::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fbinary-extensions%2F-%2Fbinary-extensions-2.3.0.tgz
      ├─ js-tiktoken@npm:^1.0.12 → npm:1.0.12::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fjs-tiktoken%2F-%2Fjs-tiktoken-1.0.12.tgz
      ├─ js-yaml@npm:^4.1.0 → npm:4.1.0::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fjs-yaml%2F-%2Fjs-yaml-4.1.0.tgz
      ├─ jsonpointer@npm:^5.0.1 → npm:5.0.1::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fjsonpointer%2F-%2Fjsonpointer-5.0.1.tgz
      ├─ langchainhub@npm:~0.0.8 → npm:0.0.11::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Flangchainhub%2F-%2Flangchainhub-0.0.11.tgz
      ├─ langsmith@npm:~0.1.30 → npm:0.1.36::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Flangsmith%2F-%2Flangsmith-0.1.36.tgz
      ├─ ml-distance@npm:^4.0.0 → npm:4.0.1::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fml-distance%2F-%2Fml-distance-4.0.1.tgz
      ├─ openapi-types@npm:^12.1.3 → npm:12.1.3::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fopenapi-types%2F-%2Fopenapi-types-12.1.3.tgz
      ├─ p-retry@npm:4 → npm:4.6.2::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fp-retry%2F-%2Fp-retry-4.6.2.tgz
      ├─ uuid@npm:^9.0.0 → npm:9.0.1::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fuuid%2F-%2Fuuid-9.0.1.tgz
      ├─ yaml@npm:^2.2.1 → npm:2.4.5::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fyaml%2F-%2Fyaml-2.4.5.tgz
      ├─ zod-to-json-schema@npm:^3.22.3 → npm:3.23.1::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fzod-to-json-schema%2F-%2Fzod-to-json-schema-3.23.1.tgz
      └─ zod@npm:^3.22.4 → npm:3.23.8::__archiveUrl=https%3A%2F%2F<redacted>%3A8081%2Fartifactory%2Fapi%2Fnpm%2F<redacted>%2Fzod%2F-%2Fzod-3.23.8.tgz

Plaform: mac Node version: v20.15.0 Yarn version: 4.0.2

zacharyfmarion avatar Aug 22 '24 05:08 zacharyfmarion

The assertion is commented out now. Please update to the latest.

https://github.com/kohya-ss/sd-scripts/blob/a61cf73a5cb5209c3f4d1a3688dd276a4dfd1ecb/flux_train_network.py#L46

kohya-ss avatar Aug 27 '24 14:08 kohya-ss

sss managed to get passed that error, now getting this @kohya-ss

DarkViewAI avatar Aug 27 '24 14:08 DarkViewAI

If you use bmaltais' GUI repository, it may not have been updated yet.

kohya-ss avatar Aug 27 '24 22:08 kohya-ss