Template versions

[JuroOravec]

Writing this to compile the info I've learnt about how to step-wise make Django templates safer, faster, enable linter support, etc.

Context

It all stems from 2 issues:

1. Django templates has limited syntax.

   • Compared to modern frontend frameworks, Django templates feels sluggish. E.g. if I want to pass a value and just invert it (not x), I have to either define a filter that does just that, or find where I passed in the value to the template and update it there. If I want to use both x and not x forms in a single template, I can either use the filter, or define 2 variables (x and not_x).
   • Filters and template tags are not typed. Whatever values you pass through the template, the IDE will NOT help you.

2. Django's template tags {% ... %} have too loose definition. It's not possible / easy to write a linter that would report errors when using custom template tags, because template tags have absolute freedom in how they process their arguments.

Now Django templates could be optimized in various ways, e.g. to avoid re-rendering parts of the template that are clearly static (e.g. {{ "I am string" }}). To achieve that:

1. Django parser would need a better resolution.
2. Django template tags would have to have predictable signature.

The change to predictable template tags will be a breaking change. That's why initially I was thinking of these in the context of when django-components as a library would reach next major versions v2 and v3.

• Version 2 mentioned in https://github.com/django-components/django-components/issues/1141
• Version 3 mentioned in https://github.com/django-components/django-components/issues/1004#issuecomment-3521869570

Other relevant issues:

• https://github.com/django-components/django-components/issues/1178

But over the last 2-3 weeks I realized that this can be tackled independently of the version of django-components itself. Instead, it could be set on the Component as a "template version":

class MyTable(Component):
    version = 1  # 1, 2, 3
    template = """
      <div>
        {% component "button" %}
          Delete table
        {% endcomponent %}
      </div>
      <table>
        <tbody>
          ...
        </tbody>
      </table>
    """

Overview

Note that the "version" applies ONLY to the component's template (Component.template or Component.template_file).

It does NOT propagate.

So if ComponentA is set to version = 2, and A's template references Component B, B could still have version = 1.

Version 1

This is the current, default, version. Things work the same way as they do.

• The overall template is parsed by Django
• We only have the ownership of the {% component %} template tags (and other our tags like {% slot %}, {% fill %}, etc).
• Our own template tags support extended syntax like literal lists and dicts, e.g. {% table data=[1, 2, 3] / %}
• But other template tags or {{ }} are handled normally and do NOT allow this extra syntax

All template tags work with version 1 templates.

Version 2

In this case WE parse the overall template with our parser. This means that:

• Special syntax can be used in {{ }} or other template tags, e.g. {{ [1, 2, 3] }}, or {% for num in [1, 2, 3] %}
• Linter can be built for templates version 2 and 3.
• Can detect parts of the template that never change (e.g. {{ "I never change" }}), which can improve render speed (as mentioned in https://github.com/django-components/django-components/issues/1141)
• Will allow smarter caching, so that slots are cached based on the variables used inside the slots.

However, the price for ths is that only template tags built on top of our BaseNode will work with Version 2 templates.

That means that we'd re-define Django's built-in template tags on top of BaseNode.

As for third party template tags, these would have to be migrated by our users.

class MyList(Component):
    version = 2
    template = """
      <ul>
        {% for item in [1, 2, 3] %}
          <li>
            {{ item }}
          </li>
        {% endfor %}
      </ul>
    """

Version 3

Version 3 would be the most optimization-friendly and independent of Django. The idea is that when this project splits from Django, the template of the components would be compatible with version 3.

Thus this gives a clear path for people how to migrate from django-components to standalone components package:

djc_v1 -> djc_v2 -> djc_v3 -> components_v1

Read about v3 syntax here https://github.com/django-components/django-components/issues/1004#issuecomment-3521869570.

In a nutshell, the template syntax could be significantly simplified, as everything before could be expressed as Python expression:

• Filter {{ 3|add:1 }} -> {{ 3 + 1 }}
• Translations from _() as its own data type {{ _("my_str") }} to _() as a custom function.
• Template tags {% lorem w 2 %} -> either reimplement as components <Lorem type="w" count=2 /> or call as function {{ lorem("w", 2) }}.

V3 templates would allow only components. No filters, but also no template tags. And components would be defined with </> instead of {%...%}.

By getting rid of {% ... %} syntax, we'll be able to treat HTML tags (e.g. <div>) as first class objects in the template.

What this means is that in v3:

• Templates will have to be valid HTML.
  • Note: In v1 and v2, we check for correctness only once the template is rendered. But the template could otherwise be incomplete HTML, e.g. <div> {% end "div" %}
• The linter will be able to offer suggestions for autocompletion for HTML tags.

class MyList(Component):
    version = 3
    template = """
      <ul>
        <li for="item in [1, 2, 3]">
          {{ item}}
        </li>
      </ul>
    """

Conclusion

The bottom line is that all these 3 versions will be able to live simultaneously one next to each other, even if django-components as a library is on v0.XX or v1, etc.

[EmilStenstrom]

I think this sounds like a great plan. Using versions makes sure there's an upgrade path, like you say.

My only comment is the code example in the version 3 block. You write that we'll be able to treat html tags as first class objects in the template, and then use the <li for="item in [1, 2, 3]"> example. Is <li> a component in this example? But <div> is not?

[jonathan-s]

Just adding a data point here, I think there might be some adoption resistance if the syntax change (too much) in later versions. In other words having a drop-in replacement most definitely have a lot of value. I'm sure you're also aware of https://github.com/LilyFirefly/django-rusty-templates which is an ongoing effort (currently don't know how far it's gotten though).

[JuroOravec]

My only comment is the code example in the version 3 block. You write that we'll be able to treat html tags as first class objects in the template, and then use the

example. Is

a component in this example? But

is not?

@EmilStenstrom Still to be ironed out. But in that example <li> is still just plain HTML tag. I'm leaning towards using c: prefix to identify components, e.g. <c:MyTable>. So <li> or even <MyTable> would be treated as HTML, while <c:li> and <c:MyTable> as components.

My only comment is the code example in the version 3 block. You write that we'll be able to treat html tags as first class objects in the template, and then use the

example. Is

a component in this example? But

is not?

@jonathan-s I guess by drop-in replacement you mean replacement for the default Django template engine? This has been my hunch for some time, but I will yet have to explore how / whether it's possible. Another possibility is that the DJC system could work completely embedded in the original Django template engine.

So whether django-components would be installed as separate engine or within Django template engine, you could still switch beteen v1-v3 for the templates owned by components.

---

Btw thanks for reminding me of LilyFirefly/django-rusty-templates. I want to get in touch with them once I have something more tangible to show.

[jonathan-s]

I guess by drop-in replacement you mean replacement for the default Django template engine?

Well, what I mean is that if write a template for a django-component (however I write that, or whether it is a completely separate engine or exists within django engine). I would expect that the syntax is not wildly different. I would say it would be a mistake (in my opinion), to drop the django syntax (at any point). But I guess I would be ok if the syntax is extended.

you can do this and this, if you use this other syntax, because django template syntax isn't enough.

That would feel better at least for me.

I'm also guessing, if you want any real perf improvements you actually need to write the engine. I think you should also check out their code and get a feel for what they're doing. :)

[JuroOravec]

I'm also guessing, if you want any real perf improvements you actually need to write the engine.

Yup, that's true. To clarify, I am indeed writing that kind of engine that you are thinking of. As in some sort of system that takes a template, figures out what's in there, and is also able to render it.

When I said Django template engine in the previous message I meant the interface that one sets in TEMPLATE.BACKEND setting. (The interface is defined in django.template.backends.base)