dify icon indicating copy to clipboard operation
dify copied to clipboard

fix(template-transform): use base64 encoding for Jinja2 templates to fix #26818

Open devbyteai opened this issue 3 weeks ago • 1 comments

Summary

Fixes #26818 - Textarea pre-fill values in Template node were not displaying correctly when using Jinja2 variables.

Problem

When creating HTML forms in Template node with Jinja2 variables:

  • <input value="{{ task.get('Task ID', '') }}"/> - WORKED
  • <textarea>{{ task.get('Issues', 'No issues') }}</textarea> - DIDN'T WORK

Root Cause Analysis

The Jinja2 template was embedded directly into the generated Python script using triple-quoted strings:

template = jinja2.Template('''<textarea>{{ content }}</textarea>''')

If the template contained ''', single quotes, double quotes, or multiline content (common in textareas), the generated Python script would have a syntax error.

Interestingly, the inputs were already safely handled via base64 encoding:

inputs_obj = json.loads(b64decode('eyJjb250ZW50IjogInRlc3QifQ==').decode('utf-8'))

But the template code was inserted via direct string replacement without any encoding protection.

Solution

Apply the same base64 encoding pattern to templates that is already used for inputs:

  1. Added serialize_code() method to TemplateTransformer base class for base64 encoding template code

  2. Override assemble_runner_script() in Jinja2TemplateTransformer to encode template as base64 before embedding in script and decode at runtime

Before (breaks with special chars):

template = jinja2.Template('''<textarea>{{ content }}</textarea>''')

After (works with any content):

template_code = b64decode('PHRleHRhcmVhPnt7IGNvbnRlbnQgfX08L3RleHRhcmVhPg==').decode('utf-8')
template = jinja2.Template(template_code)

Scope & Impact Assessment

Jinja2 templates only: I verified that Python3 and JavaScript transformers are NOT affected. They insert user code at the top level of the script (not inside a string literal), so special characters in user code are handled correctly by the respective language parsers.

Backward compatible: No changes to external APIs or workflow behavior. The fix is purely internal to how templates are embedded in generated runner scripts.

Testing

I added comprehensive regression tests to ensure this issue doesn't recur:

test_jinja2_template_with_special_characters Tests a template containing all the problematic characters: triple quotes ''', single quotes, double quotes, and multiline content. This template would have broken the old implementation. The test verifies that HTML output is correctly rendered with all special characters intact.

test_jinja2_template_with_html_textarea_prefill Direct reproduction of issue #26818 - an HTML textarea element with Jinja2 variable content containing newlines and quotes. Verifies the exact scenario from the bug report works correctly.

test_jinja2_assemble_runner_script_encodes_template Unit test verifying that assemble_runner_script() correctly base64-encodes the template. Asserts that the encoded template appears in the script and the raw template does NOT appear (proving encoding is working).

Updated existing tests test_jinja2 and test_jinja2_get_runner_script were updated to use the new _template_b64_placeholder pattern with proper base64 encoding.

Files Changed

  • api/core/helper/code_executor/template_transformer.py - Added serialize_code() method for base64 encoding
  • api/core/helper/code_executor/jinja2/jinja2_transformer.py - Override assemble_runner_script() to use base64 encoding
  • api/tests/integration_tests/.../test_code_jinja2.py - Updated tests + added regression tests
  • api/tests/test_containers_integration_tests/.../test_code_jinja2.py - Updated tests + added regression tests

devbyteai avatar Dec 26 '25 18:12 devbyteai

Summary of Changes

Hello @devbyteai, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug where Jinja2 templates failed to render correctly when they contained special characters, particularly within HTML textarea elements. The core problem stemmed from templates being directly embedded as string literals in the generated Python script, leading to syntax errors. The solution involves implementing base64 encoding for the template content, ensuring that any characters can be safely embedded and decoded at runtime. This change significantly improves the robustness of Jinja2 template processing, preventing unexpected failures and enhancing the reliability of template-based functionalities.

Highlights

  • Problem Resolution: Fixed an issue where Jinja2 templates containing special characters (like quotes or newlines) would cause syntax errors in the generated Python runner script, specifically impacting textarea pre-fill values.
  • Encoding Implementation: Introduced base64 encoding for Jinja2 template code, mirroring the existing safe handling of input data, to prevent syntax errors during script generation.
  • Architectural Change: A new serialize_code() method was added to the TemplateTransformer base class, and the assemble_runner_script() method in Jinja2TemplateTransformer was overridden to apply this encoding and subsequent decoding at runtime.
  • Scope & Compatibility: The fix is specific to Jinja2 templates, does not affect Python3 or JavaScript transformers, and is fully backward compatible with no external API changes.
  • Robust Testing: Comprehensive regression tests have been added to validate the fix for special characters, HTML textarea pre-fill, and to ensure correct base64 encoding within the runner script.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

[^1]: Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

gemini-code-assist[bot] avatar Dec 26 '25 18:12 gemini-code-assist[bot]