Call order regarding "tools" and "contexts"

[Ethereal-sakura]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

Provide a description of the help you need

我的使用场景如图所示，我的知识库里面第一条就是我提问问题的答案，但是模型仍然是调用了工具来进行搜索 How does the model distinguish when to call the context from the tool？

[dosubot[bot]]

🤖

嘿，@HelloEveryonehh！很高兴再次见到你。希望一切都好！

I'm sorry, but I'm unable to view the images you've attached. However, I can provide some information based on your question.

The dify model doesn't directly determine when to retrieve information from the context versus using the tool. Instead, it's the responsibility of the ToolModelManager to manage the interaction between the model and the tools.

The ToolModelManager class has several methods that interact with the model and the tools. For example, the get_max_llm_context_tokens method retrieves the maximum number of tokens that the model can handle in its context. The calculate_tokens method calculates the number of tokens in the prompt messages and model parameters.

The invoke method is where the model is actually invoked with the parameters in the user's own context. This method takes in several parameters including the user id, tenant id, tool type, tool name, and prompt messages. It then retrieves the model instance and credentials, calculates the number of prompt tokens, and invokes the model with these parameters.

The invoke method also handles the creation of a ToolModelInvoke instance, which records the details of the model invocation. This includes the user id, tenant id, tool type, tool name, model parameters, prompt messages, and model response.

You can find more details in the ToolModelManager source code.

As for handling different expressions of the same query in different languages, the dify platform uses a language mapping system. This system is defined in the languageMaps and I18nText objects, which map language codes to their respective languages.

For example, the I18nText object maps language codes to strings, allowing for different expressions of the same text in different languages. Similarly, the NOTICE_I18N object contains different expressions of the same notice in different languages.

The getModelRuntimeSupported function is used to get the model runtime supported for a given locale. If the locale is 'zh-Hans', it replaces the '-' with '', otherwise, it returns the first language supported ('en-US') with '-' replaced by ''.

Here is the relevant code:

export const languageMaps = {
  'en-US': 'en-US',
  'zh-Hans': 'zh-Hans',
  'pt-BR': 'pt-BR',
  'es-ES': 'es-ES',
  'fr-FR': 'fr-FR',
  'de-DE': 'de-DE',
  'ja-JP': 'ja-JP',
  'ko-KR': 'ko-KR',
  'ru-RU': 'ru-RU',
  'it-IT': 'it-IT',
}

export type I18nText = {
  'en-US': string
  'zh-Hans': string
  'pt-BR': string
  'es-ES': string
  'fr-FR': string
  'de-DE': string
  'ja-JP': string
  'ko-KR': string
  'ru-RU': string
  'it-IT': string
}

export const getModelRuntimeSupported = (locale: string) => {
  if (locale === 'zh-Hans')
    return locale.replace('-', '_')

  return LanguagesSupported[0].replace('-', '_')
}

You can find more details in the language.ts source code.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• Using a different Chinese expression fails to match the context.
• 配置了上下文，提问方式换一下，就无法命中上下文
• api/core/tools/model/tool_model_manager.py
• web/utils/language.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Yash-1511]

Hey @HelloEveryonehh, you can solve this problem in this ways. First, write clear and crystal description of your knowledge base. Second, use prompt engineering to know when to use tools or your own knowledge. Using both methods should help you get the result you want.

• How to write good knowledge description
• Agent Assistant prompt guide

[crazywoola]

This highly depends on your description and the way you describe the knowledge. Also it depends on llm itself. If a llm is not that good to understand or misunderstand your question. It will use tools or knowledge base to answer your question randomly.

[dosubot[bot]]

Hi, @HelloEveryonehh,

I'm helping the team manage our backlog and am marking this issue as stale.

It looks like the issue was raised to seek clarification on the call order regarding "tools" and "contexts" in a specific scenario. I provided a detailed response explaining the role of the ToolModelManager class and the language mapping system used in the dify platform. User "Yash-1511" suggested writing a clear knowledge base description and using prompt engineering to address the issue, while user "crazywoola" emphasized the importance of the knowledge description and the capabilities of the language model in determining when to use tools or the knowledge base.

Could you please confirm if this issue is still relevant to the latest version of the repository? If it is, please let the team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!