feat: Jinja2 template engine in workflow templates

[korolenkowork]

Closes #4594

📑 Description

This PR added the whole power of Jinja2 templating to the KeepHQ workflow

✅ Checks

• [x] My pull request adheres to the code style of this project
• [x] I have updated the documentation as required
• [x] All the tests have passed

✨ Features

• Users can now choose their preferred template engine for workflows: Jinja2 or Mustache
• Mustache remains the default template engine
• Introduced a syntax validator that raises a RenderError if invalid syntax is used for the selected engine
• If an incorrect templating engine is specified in the YAML, a ValueError is raised

🛠 Improvements

• Mustache (via Chevrone) now uses a tracking dictionary to log missing keys in the context, implemented in a thread-safe way.

⚠️ Known Limitations

• During Jinja2 rendering, only missing top-level keys can be detected — full key paths cannot be tracked. Example: {{ alert.namespase }} will report {{ namespace }} as missing, but due to how Jinja2's Undefined class works, it cannot identify the full path to the missing value.

• It’s not possible to use a tracking dictionary for Jinja2 in the same way as Mustache, because Jinja2 internally converts the context to a plain dict during rendering.

[vercel[bot]]

@EnotShow is attempting to deploy a commit to the KeepHQ Team on Vercel.

A member of the Team first needs to authorize it.

[shahargl]

@EnotShow lot of unit tests are failing, can you fix that? :)

[talboren]

Also a quick question about backwards compatibility - not sure if I understand if this will still support mustache or we're replacing to Jinja completely? Maybe we should support templating=mustahce or templating=jinja in the top of the workflow to decide which engine we want to use. Will the keep. functions work?

[korolenkowork]

@talboren

1. This PR fully replaces Chevron with Jinja2 in workflows. I’ve proposed sticking with it, since once this is done, the previous workflows need to remain fully compatible. But it makes sense to proceed cautiously, especially since we don’t yet know all the potential issues this might introduce. So if the team decides it has to be done, I’m happy to go ahead with it.

2. Yes, keep functions working perfectly!

[korolenkowork]

Ahh, now that I've gone through the Chevron docs, I see why Mustache needs to stay

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 54.81928% with 75 lines in your changes missing coverage. Please review.

Project coverage is 46.72%. Comparing base (d39afd3) to head (fe36005). Report is 25 commits behind head on main.

Files with missing lines	Patch %	Lines
keep/iohandler/iohandler.py	53.06%	69 Missing :warning:
keep/workflowmanager/workflow.py	50.00%	4 Missing :warning:
keep/conditions/base_condition.py	50.00%	1 Missing :warning:
keep/parser/parser.py	85.71%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4595      +/-   ##
==========================================
- Coverage   46.77%   46.72%   -0.05%     
==========================================
  Files         164      165       +1     
  Lines       16841    17125     +284     
==========================================
+ Hits         7877     8002     +125     
- Misses       8964     9123     +159     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[korolenkowork]

I need some help with workflow execution tests for jinja2 cose, I not so sure about how it work, but except it work seems like done

[korolenkowork]

Done!

[talboren]

@EnotShow i'll review the PR asap. a bit busy the last couple of days but excited about it 🤩

[korolenkowork]

I’m encountering a strange error in my tests that I can’t reproduce locally. The error message is:

FAILED tests/e2e_tests/incidents_alerts_tests/test_filtering_sort_search_on_alerts.py::test_alerts_stream - AssertionError: Locator expected to have count '20'
Actual value: 30 
Call log:
  - LocatorAssertions.to_have_count with timeout 5000ms
  -   - waiting for locator("[data-testid='alerts-table'] table tbody tr")
  -     9 × locator resolved to 30 elements
  -       - unexpected value "30"
============ 1 failed, 32 passed, 15 warnings in 161.66s (0:02:41) =============

The test expects 20 alerts, but it’s finding 30. Interestingly, when I run the test locally, it passes with exactly 20 alerts.

Could the issue be that the database is not being recreated properly for every test run?

[talboren]

I’m encountering a strange error in my tests that I can’t reproduce locally. The error message is:

FAILED tests/e2e_tests/incidents_alerts_tests/test_filtering_sort_search_on_alerts.py::test_alerts_stream - AssertionError: Locator expected to have count '20' Actual value: 30 Call log:

• LocatorAssertions.to_have_count with timeout 5000ms
• • waiting for locator("[data-testid='alerts-table'] table tbody tr")
• • 9 × locator resolved to 30 elements
• • • unexpected value "30" ============ 1 failed, 32 passed, 15 warnings in 152.97s (0:02:32) =============

The test expects 20 alerts, but it’s finding 30. Interestingly, when I run the test locally, it passes with exactly 20 alerts.

Could the issue be that the database is not being recreated properly for every test run?

It's a known issue caused by some PR @skynetigor pushed. He's on it :)

[korolenkowork]

Ok, so it's ready for review! :)

[skynetigor]

Hi @EnotShow! Now it looks good. Just a quick question: Mustache works as well as before and is unchanged in terms of functionality it provides, right?

[korolenkowork]

Hey @skynetigor, The main goal was to preserve Mustache’s functionality and make Jinja2 behave the same way as Mustache. So yes, Mustache continues to work just as it did before.

[korolenkowork]

Hi @talboren, @shahargl! All tests are passed, and I received approval from @skynetigor, so it seems ready for merge!

[shahargl]

@EnotShow let me know when you want the final review + merging

[korolenkowork]

@talboren This is ready for review, but I ran into an issue with Jinja2.

Jinja2 doesn't allow hyphens in identifier names when accessed as attributes in templates. To work around this, I implemented a function which replaces code like:

Pod status report:
{% for pod in steps.get-pods.results %}
Pod name: {{ pod.metadata.name }} || Namespace: {{ pod.metadata.namespace }} || Status: {{ pod.status.phase }}
{% endfor %}

with:

Pod status report:
{% for pod in steps['get-pods'].results %}
Pod name: {{ pod.metadata.name }} || Namespace: {{ pod.metadata.namespace }} || Status: {{ pod.status.phase }}
{% endfor %}

This change uses dictionary-style access, which is valid in Jinja2 for keys that contain hyphens. It seems to work, but I not sure if this solution is proper enough. What you think?

[talboren]

@talboren This is ready for review, but I ran into an issue with Jinja2.

Jinja2 doesn't allow hyphens in identifier names when accessed as attributes in templates. To work around this, I implemented a function which replaces code like:

Pod status report:
{% for pod in steps.get-pods.results %}
Pod name: {{ pod.metadata.name }} || Namespace: {{ pod.metadata.namespace }} || Status: {{ pod.status.phase }}
{% endfor %}

with:

Pod status report:
{% for pod in steps['get-pods'].results %}
Pod name: {{ pod.metadata.name }} || Namespace: {{ pod.metadata.namespace }} || Status: {{ pod.status.phase }}
{% endfor %}

This change uses dictionary-style access, which is valid in Jinja2 for keys that contain hyphens. It seems to work, but I not sure if this solution is proper enough. What you think?

makes absolute sense! I'll try to review it ASAP.

[cursor[bot]]

🚨 BugBot couldn't run

Pull requests from forked repositories are not yet supported (requestId: serverGenReqId_06c76e83-cbf9-46fb-b4f7-80acc0b782b5).

[korolenkowork]

@shahargl @talboren any updates on this?

[shahargl]

@korolenkowork see comment on the other PR. This is crazy great but I need to see how I prioritize reviewing it.

[korolenkowork]

Hey, @shahargl does it means that changes permanently declined?