Jinja in LLM node

[l544301590]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.7.2

Cloud or Self Hosted

Cloud

Steps to reproduce

1. Use an LLM node in workflow with Jinja enabled

2. Mock the variable segments

3. Run the workflow and check the input of LLM node

---

Here is the DSL file: abc.yml.txt

✔️ Expected Behavior

❌ Actual Behavior

[dosubot[bot]]

To resolve the issue where the LLM node with Jinja enabled does not produce the expected output in Dify version 0.7.2, follow these steps:

1. Enable Jinja-2 Templates: Ensure that the Jinja-2 template language is enabled in the advanced settings of the LLM node.

2. Correct Template Syntax: Verify that the Jinja-2 template syntax used in your prompts is correct. Refer to the official Jinja documentation for guidance on proper syntax and usage.

3. Variable Insertion: Ensure that you are correctly inserting variables into the prompt. You can call out the variable insertion menu by typing "/" or "{" to insert special variable blocks or upstream node variables into the prompt as context content.

Example of a Jinja-2 template in the prompt editor:

{% for segment in segments %}
    {{ segment.content }}
{% endfor %}

4. Context Variables: If you are using context variables, ensure they are properly configured and associated with the correct upstream node variables. Context variables are used to insert external text content retrieved from other nodes into the prompt.

5. Fetch Jinja Inputs: Ensure that the _fetch_jinja_inputs method in the LLMNode class is correctly fetching and processing Jinja inputs from the variable pool. This method should correctly parse and handle different types of variables (strings, lists, dictionaries, etc.) and integrate them into the prompt.

6. Model Configuration: Ensure that you have selected the appropriate model and configured its parameters correctly. Dify supports various models like OpenAI's GPT series, Anthropic's Claude series, and Google's Gemini series. Choose a model based on its inference capabilities, cost, response speed, and context window.

7. Advanced Settings: Utilize advanced settings such as memory, memory window, and conversation role name settings to enhance the LLM's understanding and response accuracy.

8. Prompt Generator: If you are struggling to create effective prompts, use the prompt generator feature to quickly generate suitable prompt words for your business scenario.

By following these steps, you should be able to resolve the issue and achieve the expected output from the LLM node with Jinja enabled ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[MH4GF]

@crazywoola I am interested in contributing to this issue😄 can I submit a PR?

[crazywoola]

Yes, of course you can.

[MH4GF]

🏃🏻‍♂️📝

• Reproduce this situation and add a test that fails: https://github.com/langgenius/dify/pull/8088/commits/9f58060fada03449bb8a27868c8cba99adb46cb9
  • https://github.com/langgenius/dify/actions/runs/10757225351/job/29831027433?pr=8088#step:13:243
• The CodeExecutor in the test code has been replaced by a mock, but in the case of Jinja2 the library is executed directly, so the results are probably the same as in the production environment.

https://github.com/langgenius/dify/blob/b1918dae5ecc4270266dcbcfad493f265d5f907a/api/tests/integration_tests/workflow/nodes/__mock/code_executor.py#L20-L21

[crazywoola]

🏃🏻‍♂️📝

• Reproduce this situation and add a test that fails: 9f58060

  • https://github.com/langgenius/dify/actions/runs/10757225351/job/29831027433?pr=8088#step:13:243

• The CodeExecutor in the test code has been replaced by a mock, but in the case of Jinja2 the library is executed directly, so the results are probably the same as in the production environment.

https://github.com/langgenius/dify/blob/b1918dae5ecc4270266dcbcfad493f265d5f907a/api/tests/integration_tests/workflow/nodes/__mock/code_executor.py#L20-L21

@laipz8200 Could you take a look at this?

[MH4GF]

Apparently, this is because [{'id': '1', 'content': 'abc'}, {'id': '2', 'content': 'def'}] is being passed to Jinja2 inputs as a string rather than an array.

Also https://j2live.ttl255.com/, when value was set to a string, the result was similarly just a line break.

---

It would seem that a process to parse to an array would be a good way to do this, but that is a risky action and it seems difficult for me, as a newcomer, to make a decision. After this, it is left to the maintainer! @crazywoola ( @laipz8200 )