EmilStenstrom
Support for partials
Hello,
Not really a bug but a feature request.
Wonderful experience working with 'django-components' this far, I know that this has been asked already, a lot. I think most users of this package are using with in conjunction with 'htmx' for dynamic user interfaces, html over the wire; partials, the ability to refere to a specific part of a template file would be great addition to this library, it would certainly reduce boilerplate, i just want to point to the following example, which i think will be a pattern among user applications:
urls.py
def list_clients(request):
if request.htmx:
template_name = "clientlist/clientlist.html"
else:
template_name = "clientlist/clientlist_page.html"
return render(request, template_name)
urlpatterns = [
path('', list_clients, name='list_clients'),
...
clientlist_page.html
{% extends "page.html" %}
{% block content %}
{% include "clientlist/clientlist.html" %}
{% endblock content %}
clientlist.html
{% component "objlist_view" / %}
A very similar example can be found on 'django-htmx':, ('partial rendering'), the need to deliver complete 'html documents' to url_resolvers, vs 'html components' to requests is real. The current approach I am using works but i feel it would be cleaner if I could put the whole html inside a single file and refer only to the component part on demand, I will be only appending boilerplate files as the application grows.
I want to be able to use template inheritance, feel like the approach taken by 'iwanalabs' on the matter, using 'includes' is rather hacky, and will be a limitation as complexity grows. Are people approaching this problem in any other way? Also, I can only assume that this package is incompatible with 'djanto-template-partials', right? That could also be a workaround.
Thank you. Best regards.
Hi @ramalhoruben, great to hear such praise, thanks!
A very similar example can be found on 'django-htmx':, ('partial rendering'), the need to deliver complete 'html documents' to url_resolvers, vs 'html components' to requests is real. The current approach I am using works but i feel it would be cleaner if I could put the whole html inside a single file and refer only to the component part on demand, I will be only appending boilerplate files as the application grows.
Could you elaborate this a bit more, maybe with some examples?
At first read it sounded similar to this idea https://github.com/EmilStenstrom/django-components/issues/635. But on second read it sounds like your idea is more on handling the URL for components. And on the third read, I'm not sure which one it is š
When it comes to managing URLs, django-components allows the Components to be used as Django Views (see docs):
from django.urls import path
from components.calendar.calendar import Calendar
urlpatterns = [
path("calendar/", Calendar.as_view()),
]
Does this address your issue, or do you have something else in mind?
I want to be able to use template inheritance, feel like the approach taken by 'iwanalabs' on the matter, using 'includes' is rather hacky, and will be a limitation as complexity grows. Are people approaching this problem in any other way?
Could you link to some examples? To give me an ide what you mean by the "includes".
Also, I can only assume that this package is incompatible with 'django-template-partials', right?
Don't know if anyone tried. But if django-template-partials uses template tags to insert the partials, then it should be possible to insert django-component into django-template-partials and vice versa.
@ramalhoruben Does the below correspond to what you are asking?
In the ideal world, on the first page load you need to load the full page, but on all later loads, you just want to load the things that change when clicking around. This means that each view needs to handle two things, the first load and the component update.
One way of doing this is that for each view, you tell it how to render the first time (full page load), and then how to render updates (via HTMX and components). That's the pattern I see described above. That's a nice pattern because it minimizes the amount of javascript needed on a page, and first loads are fast (because entirely server-side) and subsequent ones just reloads what's changed.
You're asking for a way to streamline the above pattern for a site with lots of views, where you don't want to say the same thing over and over for each view.
Hi,
@EmilStenstrom exactly the problem and pattern I tried to describe, I wanted to avoid having having templates for both components (fragments) and a component pages (full html pages), in case the user issues a full page load, as both versions might be served from same url, depending on request headers.
Meanwhile I came up with the following utility code to workaround the problem:
urls.py
path('', main_component, {'component_name':'clients_list'}, name='clients'),
utility:
def main_component(request, component_name):
if request.htmx:
comp = django_components.registry.get(component_name)
response = comp.as_view()(request)
else:
context = {'component_name': component_name}
response = render(request, 'main.html', context)
response['vary']='HX-Request'
return response
main.html
{% block content %}
<main id='router-content' class="y-content container mb-4" hx-boost="true">
{% component "dynamic" is="{{component_name}}" / %}
</main>
{% endblock content %}
@JuroOravec thank you for your answer.
I thought a bit more about this. @ramalhoruben Why do you want to serve the fragment from the same URL? I can imagine that might be because you're using REST-like URL patterns, which doesn't allow to specify HTML fragments other than via HTTP methods or query data.
E.g. If I had a route like /projects/13, and I wanted to have a fragment for that page, then, normally, something like /project/13/modal should return a separate page, NOT a fragment of page /project/13/modal.
If that's the case, then I wonder if we could make it easier by managing a set of URL that would render the components.
E.g. imagine that I'm on the page /projects/13 and I want to fetch the component MyModal and insert it as a fragment.
Instead of creating a new route in urlpatterns, I would just allow the component to publically accessible via an endpoint:
class MyModal(Component):
public_url = True
template = "<div></div>"
...
And to generate a URL for a component which has public_url = True, I could use get_component_url(Component). Example with HTML:
from django_components import get_component_url
from myapp.components.mymodal import MyModal
class MyPage(Component):
def get_context_data(self):
modal_url = get_component_url(MyModal)
return {"modal_url": modal_url}
template = """
<html>
<head>
<script src="https://unpkg.com/[email protected]"></script>
</head>
<body>
<div id="target">OLD</div>
<button id="loader" hx-get="{{ modal_url }}" hx-swap="outerHTML" hx-target="#target">
Click me!
</button>
</body>
</html>
"""
The true URL of modal_url would be something like /components/component/a1b3cf. And behind the scenes, it would still do the same:
path("a1b3cf/", MyModal.as_view()),
Except with the difference that one wouldn't have to have the dilemma of how to design the URLs with respect to fragments on the page.
And since we would use MyModal.as_view(), then one would still be able to configure how the component should behave when fetched, via get(), post() and other methods:
class MyModal(Component):
public_url = True
template = "<div></div>"
def get(self, request):
return self.render_to_response(
kwargs={
"myarg": request.GET["myarg"],
},
)
I've used this pattern before (not with django-components), and for me it goes like this:
- For each page you have one big component representing the body of the page. All headers and sidebars remain the same.
- Each user-facing URL represents the body content on that page, so it makes sense to share the URL between fragment and public URL.
- Now if you just make sure to build a 1-to-1 mapping between component and URL, you can get away with very little JS.
@EmilStenstrom Could you add some (pseudo)code example? Are you saying that once the page was loaded, then different URLs loaded only the body of the page and replaced only that? How did you then load the initial page? Or was it like @ramalhoruben mentioned, that you used e.g. a query param (e.g. ?body=1) to configure whether to load the full HTML document vs only the fragment?
So the full idea is this:
- The first load is always fully from the server, no JS needed. Django is rendering everything. Super quick and no stutter.
- As part of the first load you load some JS that hijacks all links to go via JS (whatever language) instead of triggering reload.
- When interpreting requests on the server, you check request.is_ajax() to decide if to send the full page or just the component as response.
Advantages:
- Fast Initial Load: Django pre-renders pages for a quick first paint.
- Seamless Navigation: JS intercepts links to fetch only needed components, avoiding full reloads.
- Progressive Enhancement: Site works without JS, while AJAX upgrades interactivity when available.
- SEO-Friendly: Server-side rendering provides fully rendered content to search engines.
- Maintainable: A single Django template system supports both full and partial responses via request.is_ajax().
Ah, makes sense, thanks! So if I understand right, the distinction is that in your case, there was a single URL, and it was decided on the server endpoint whether to respond wih whole HTML or just fragment. This makes sense when you have a static layout, and you want to update the "page" (actually just the body).
When I wrote my suggestion above, I was thinking more when I would need to update only a single form on the page, not the whole page. In that case, having a URLs for individual components would make more sense.
Btw, it seems that is_ajax was deprecated in Django 4.0 (see thread)
@JuroOravec Makes sense. My frontend skills are a bit dated :)
Btw. I'm still waiting for the future to happen! https://www.youtube.com/watch?v=aOk67eT3fpg (it gets interesting for django-components 15 minutes in)
If I understand the message in the talk correctly, then datastar could also be of interest which combines ideas from Alpine and HTMX and uses idiomorph for DOM-merging.
@EmilStenstrom Exactly the idea and cenario i tried to convey, no need to reload: resources, footer, navbar and other shared elements, only the main frame content needs to change most of the time.
@JuroOravec The page doens't need to be static, building a highly interactive crud interface over here with htmx and django-components, you can lookup the example at https://htmx-router.fly.dev/, not exactly django or django-components but it is same htmx recipe.
This might be a crazy, but hear me out! :)
Template page.html:
<html><body><h1>Hello</h1><p>{% component "random_number" / %}</p>{% component "copyright" / %}</body></html>
Normal render:
<html><body><h1>Hello</h1><p>Number: 7. <a href="/">Reload</a></p>Ā© 2025</body></html>
Secretly, the page would also be sent a template for how to rerender the page from js
{
"static": ["<html><body><h1>Hello</h1><p>", "</p>", "</body></html>"],
"1": "Number: 7. <a href=\"/\">Reload</a>",
"2": "Ā© 2025"
}
Partial rerender:
{
"1": "Number: 3. <a href=\"/\">Reload</a>",
"2": "Ā© 2088"
}
Now we could just take the rerendered data, patch the secret page structure, join the string, and replace the page.
And maybe this could be implemented as just a decorator on the original view? The partial_rendering decorator would check a HTTP header in the request, and if set execute the full view, but then take the context and use it to just rerender the components on the page, nothing else.
@components.partial_rendering
def startpage(request):
return render(request, "page.html")
Hi @EmilStenstrom
I also tought about something along those lines, currently i am doing it like this, in case it helps anyone. Might need to be changed a little bit to suit particular needs, still testing it.
urls.py
path('',ClientsListComponent.as_view(),name='clients'),
@component.register("clients_list")
@htmlpage_component("clients_list")
class ClientsListComponent(ObjListInstanceComponent):
...
where the decorator i came up with is:
from django.http import HttpResponse
from django.shortcuts import render
from django.views.decorators.vary import vary_on_headers
def htmlpage_component(component_name: str):
# Decorator for Django component classes that delivers both full and fragment responses
# depending on htmx headers.
def decorator(component_cls):
default_view = component_cls.as_view
@classmethod
def wrapped_as_view(cls):
def view(request, **view_kwargs):
@vary_on_headers("HX-Request", "HX-History-Restore-Request")
def wrapped_view(request, **view_kwargs) -> HttpResponse:
# Deliver fragment response for normal htmx requests
# https://htmx.org/docs/#history
if request.htmx and not request.htmx.history_restore_request:
return default_view()(request, **view_kwargs)
# Deliver complete page html for normal requests or url history traversal
else:
ctx = {
'component_name': component_name,
'kwargs': view_kwargs
}
return render(request, 'main.html', ctx)
return wrapped_view(request, **view_kwargs)
return view
# Replace the as_view method
component_cls.as_view = wrapped_as_view
return component_cls
return decorator
I'm currently in the mode of addressing smaller features, so I'm also revisiting this one. Some thoughts:
-
@ramalhoruben For your latest comment - recently we've added support for extensions. Maybe this could be somehow spun out into a separate extension for using django-components with HTMX?
Either way, I reckon your example could generalized into:
if is_htmx_fragment: # render component as is else: # do something elseFrom my perspective, I'm not too sure what to do with it, how to generalize it - how does one decide what should happen in the else branch? In your case that's where you've rendered the
'main.html'.As counter-example, in my project I have separate components for the pages, and separate components for the fragments. The fragment components can have their own routes with
Component.as_view(). And so I would only ever make HTMX requests to the fragment endpoints, and non-HTMX request to regular pages. And so in my case it just becomes the matter of managing the URLs.That's why in this comment, I was thinking more about making it easier to make URLs pointing to components directly. IMO it would then be easier to have separate "fragment" components, and separate "page" components.
-
@EmilStenstrom Had a look at the video on Phoenix LiveView, so now also your example makes more sense.
Since I'm using AlpineJS, this approach is outside of my interest, because replacing the whole chunks of HTML would kill any client-side state that's bound to the HTML elements. E.g. if I had an expansion panel open / closed, or some simple counter, whose logic is tracked only on the client, the state of that expansion panel / counter would get reset if I replaced a chunk of HTML like that.
So this approach would effectively force me to handle all interactions strictly on the server-side. And I like about my current setup (and where I want to take it) that it would allow both server-side AND client-side interaction handling to live side-by-side, allowing me to pick the right tool for the right job.
But those things aside, my hunch is that it would make more sense to have a ~specialized template tag~ or a custom component for defining the fragment boundaries, instead of treating each component as potential fragment, because:
-
A single large page may be made up of hundreds of components, but will likely have only tens of fragments. So it would be better if the merging logic would consider only "real" fragments.
-
With nested components, it may happen that some component input is derived or generated by its parent. Some input may even be sensitive. So not all input that a component accepts from an HTTP request can be provided from the client. So the fragments need some way to prepare the correct data based on the HTTP request.
So I think an API that would make more sense would be either something like:
class MyComponent(ComponentFragment): ...or
@fragment class MyComponent(Component): ...And behind the scenes, all this would do would be to mark the start and end of the fragment, maybe with HTML comments like below. This would allow the client-side to know how to chop-up the HTML.
bla bla <!-- FRAGMENT XYZ START --> <div> ... </div> <!-- FRAGMENT XYZ END -->Then, to define how a particular fragment should prepare the data, we could just reuse the View methods
get(),post(), etc:@fragment class MyComponent(Component): def get(self, request): post_id = request.GET["id"] post = get_object_or_404(Post, pk=post_id) return MyComponent.render( kwargs={"post": post} )And then lastly, the component fragment would have to be added to
urlpatterns. Or as I mentioned before, django-components could take care of the registering, and one would only need to get the component URL with something likeget_component_url(MyComponent)So that's for the fragment part. Next we would need to inject the HTML chopping-and-merging JS script into the page.
Again, this could be a decorator or a custom Component subclass:
class MyPage(ComponentFragmentPage): ...or
@fragment_page class MyPage(Component): ...All it would do would be to add the script to
Component.Media.js.
There'd defo be more nuances (e.g. how to differentiate the fragments?), but it feels like this is roughly how it could be implemented.
-
So my overall takeaway is that I think it makes sense for django-components to handle registering of Components as urls (opt-in) like I mentioned here, since that could be used in both cases that we're discussing here.
Hi @JuroOravec
To provide some additional context, in my project i have a base template that renders both a page header (navbar) and page body, where the page body is a dynamic component, whose component varies / is bound to the url.. if a non htmx call is made to the server then the server does return a complete render of the page with the correct component, on the presence of an htmx request then only the component contents are rendered,responded and refreshed into component.
You can consider what i am building to be my own version of django admin, having to provide a version of each component as a component but also as a page would lead to a lot of boilerplate that i am seeking to alleviate.. will want to focus soon on the business logic and not on the "application engine" itself.
A page refresh when navigating would also break the kind of experience i am looking forward to, what i currently have it's very interactive, seamless transition between lists, objects, actions (modals) and everything else..
I don't have much complexity on the client side, a light touch of JS where it was needed, i am doing mostly what you expect from a CRUD application and at the moment reusing the django-components logic's that it is rightly so encapsulated in my "ModelAdmin" classes.
I did a demo so you can have a better idea.. there is some complexity, dynamic components inside other dynamic components inside other components.. but I figure that is the same that happens inside frameworks like react or vue.
[Demo Removed]
Django-components is great, you guys are awesome.
Best regards,
Btw @ramalhoruben I love how snappy it feels! What are you using for the UI components like buttons, dialogs, snackbar, and such?
I'd love having a demo like this on the front page to showcase what can be built with django-components. Does the demo contain any sensitive information? And @EmilStenstrom what do you think?
Hi, @JuroOravec thank you.
The UI is just regular bootstrap 5 with some customization to the default styles, each dialog is a django component view, the get method will retrieve modal contents (render template) and POST apply the component action to the instance or collection of instances. POST will respond with an updated view of the form (validation errors, 400) which will refresh modal contents or be auto-closed on success via vanilla JS and response headers. The toast itself is not bootstrap 5 as I wanted something different.. when a POST/PUT/DELETE/PATCH request is done to the server as part of the response "HX-Trigger Response Headers" are delivered with feedback: success, warnings, errors (ValidationError's too) can be delivered this way. I just listen for this events and launch a toast everytime i get an event, i am not using the django messages framework, having it as a progressive SPA makes this much easier to implement. The backdrop/sleep will update lists and objects upon awake, except if object is currently open for modifications, in which case that component itself will remain unchanged (atm).
Yes, this demo was only made for you guys, while the information is not that sensitive I wouldn't want it to be mainstream, plan on removing it later.
Regarding the initial question about partials I wouldn't make any change to allow partials, unless a stronger counter example appears, the framework is already very powerful as it is. I think the solution to the problem, as I did, is architecture, plus something along the lines of the utility i provided.
Best regards
@ramalhoruben Very happy to see that example. That people can build powerful stuff with our framework is super inspiring. If you would like to make a version that we could share, we would be happy to. There's no better marketing than happy developers :)
@JuroOravec I really like your implementation idea. I wonder though, if we could make all components be fragments by default? Maybe we could have just a single setting that enabled the component URL:s and added the chopping HTML. I fully understand that this isn't compatible with a frontend framework; instead it's a replacement of a frontend framework with a backend one. I know many Django developers that are not super comfortable in React/Alpine/HTMX. For them to be able to get a rich client side experience without writing frontend code, would be magical.
Btw over the past week I've implemented the feature for optionally generating URLs for components with Component.Url.public, like what I mentioned here (I'll open a PR for it after https://github.com/django-components/django-components/pull/1082).
I think it makes sense to treat that and the HTML chopping approach as separate features that may be used independently. We can dedicate the rest of this discussion on the latter.
Love the name btw, HTML chomping š
TL;DR
-
IMO we don't need need to rely on the HTML chomping, because our components SHOULD output valid HTML, and SHOULD be used only in places where valid HTML tags could. This, combined with the fact that our components have the
data-djc-idattributes, means that we can make changes to the HTML document by exactly removing / replacing the elements in the DOM.So we can make more fine-graned updates to the DOM than if we joined up all strings and replaced the whole HTML.
-
Thus, for simple updates, when user needs to replace the content of a SINGLE fragment, they can easily achieve that with HTMX or similar.
-
The Elixir-like approach would only make sense in more complex updates, where we'd want to update MANY components at the same time. But for that we'd need to compute the diff of which components changed. And for that I think we'd need to keep the state of the page on the server-side / in the database (e.g. at least keep a JSON of the old inputs that were passed to the root component, and compare it with the new inputs, and derive which nested components changed based on that).
And if need to keep the render state in the database, then I think this should be addressed by libraries like tetra, not django-components.
Does the HTML chomping actually make sense?
Handling the rendered HTML as a list of static and dynamic strings IMO only makes sense if we expect that the rendered components may output invalid HTML. If we expect components to output valid HTML (which we do), IMO we can take more nuance.
What I mean - Imagine that we had a templates like this (Note: This is IMO bad design, do NOT do this):
<div {% component "attrs" class="text-red" / %}
hello!
</div>
Or
{% component "tag" "div" %}
hello!
</div>
In the first example, the component attrs would render class="text-red" >. Notice that the output of attrs includes the closing tag for the <div> start tag.
In the second example, the tag component renders the whole <div> start tag.
In both cases, the static text, without the components, is an invalid HTML, e.g.:
<div
hello!
</div>
and
hello!
</div>
If this is the case, then the approach taken by Elixir with joining of the static and dynamic parts makes sense, because you don't know where are the boundaries between static / dynamic parts in the HTML.
However, if we take approach by React and Vue, then the text MUST be a valid HTML even without the components:
So either something like this:
{% component "div" class="text-red" / %}
hello!
{% endcomponent %}
or like this:
<div>
{% component "button" class="text-red" / %}
hello!
{% endcomponent %}
</div>
I think it makes sense for django-components to assume / expect this constraint.
We already rely on it to enable the inserting of the JS / CSS variables, where we need to insert custom attributes like data-djc-id-a1b2c3.
This constraint is also implicitly enforced when one would use the HTML / django-cotton syntax. Because if you did something like this:
<div <c-attrs class="text-red" />
hello!
</div>
Then that's an invalid HTML, and you wouldn't get intellisense or proper syntax highlight.
So to summarize:
- The camp of users who prefer the django-cotton like syntax will be already formatting their components as valid HTML.
- Some of our logic, like for the inserting the JS / CSS variables, relies on the components having a valid HTML.
A better approach?
So if we assume that:
- The components output valid HTML
- The components are inserted into the template in an HTML-compatible way
Then IMO we don't need the JSON with the static / dynamic parts, and we DON'T need to join the text to make up new HTML.
{
"static": ["<html><body><h1>Hello</h1><p>", "</p>", "</body></html>"],
"1": "Number: 7. <a href=\"/\">Reload</a>",
"2": "Ā© 2025"
}
Instead, we know exactly which HTML elements in the document belong to which component. Because they will be tagged with the data-djc-id-a1b2c3 attribute.
In the following example, we know that if we wanted to replace the content of a particular component, that we'd find it by its data-djc-id attribute:
<div>
<button class="text-red" data-djc-id-a1b2c3>
hello!
</button>
</div>
We support also multi-root components, so it could be also something like below. In this case, we expect that all the roots are next to each other in the HTML.
<div>
<button class="text-red" data-djc-id-a1b2c3>
hello!
</button>
<p data-djc-id-a1b2c3>
Hint: Click the button.
</p>
</div>
So the flow would be different - we wouldn't need to join and replace the whole HTML.
Instead, we would:
- Get the
data-djc-idof the fragment that we want to update - Fetch the re-rendered component
- Only replace the content of the fragment.
But if we're updating only a single fragment at a time, and the fragment is a valid HTML, then this could all be in theory done with libraries like HTMX.
Ofc there are some nuances, e.g. this could be inefficient if one chooses to replace a component that encompases the whole page. Instead, one should choose to replace smaller, deeply nested components instead to minimize the diff.
Where the Elixir's approach gets useful and relevant is if we considered that we wanted to make updates to multiple components at the same time.
So I think a better way for me to think about this issue is to rephrase the question as:
Given that I have an HTML with components, and I make a request to the server to get updates, then how will I know what are the differences between the old and new render, and how will I replace them?
-
When it comes to "how will I replace them", that's answered above - we identify the fragments by their
data-djc-idattributes and make point replacements. -
So IMO the hard question is how to how to make a diff between the old and new content, and how to derive from that which
data-djc-idto replace with what content.- Maybe I could send the server the previous input, so the server can render both new and old, and see which components received different inputs, and make a tree of diffs based on that?
- Or maybe, to keep stuff secure, the inputs of the last render are kept on the server in the db, and I send to the server only an ID of last render.
- Maybe I could send the server the previous input, so the server can render both new and old, and see which components received different inputs, and make a tree of diffs based on that?
And so, I think this boils down to the question of "how can I make the diff on the server". And as such, I think this feature would make more sense for a library like tetra, which focuses, among other, on storing the UI state in the database, and making changes based on that.
Now, I think that Tetra could eventually become part of the django-components family / umbrella, but that's for another discussion š
Draft thoughts
Below are some ideas that I eventually discarded as I was writing this comment
Click to expand
As for the implementation, one advantage of requiring users to explicitly define fragments would be that the HTML sequence in the JSON file could be kept flat.
What I mean by that - Components may contain other components. So if we represented the HTML as shown in the video:
{
"static": ["<html><body><h1>Hello</h1><p>", "</p>", "</body></html>"],
"1": "Number: 7. <a href=\"/\">Reload</a>",
"2": "Ā© 2025"
}
Then practically we'd allow to replace only top-level components. So if the top-level components contained other components, tough luck.
And so with fragments, we could give people a condition that fragments cannot contain other fragments. This way we could achieve a flat representation like above.
However, what I like more is that, instead of having a flat structure, we'd keep a tree of static content + components:
{
"static": ["<html><body><h1>Hello</h1><p>", "</p>", "</body></html>"],
"1": ["Number: 7.", "<a href=\"/\">Reload</a>"],
"2": ["Ā© 2025"]
"1:1": ["I'm nested!"]
}
So how you'd read the JSON is that:
- "1" means "insert inside gap 1 of "static"
- "1:1" means "insert inside gap 1 of "1"
WIth this approach, I can imagine that the joining of all the parts could be get slower, since we'd have to recursively check for fills for the gaps in each of the lists.
But slow joining could be offset with caching - E.g. I can resolve the strings for "1" and "1:1", "1:2", "1:2:3", etc... And if the server updates the gap "2", I know I can still reuse "1" without rebuilding it.
But this still leaves the question - how would we know where to insert the fragment?
I'm thinking that the simplest would be to pass the info (about which fragment to replace) in the URL.
I'll go over the whole flow so as to make sense of it all:
Sending request
How is the initial request made? I might be wrong, and don't want to rewatch the video right now, so I'm guessing the flow is something like this:
-
When clicking on a link, it doesn't make the request from within the document (e.g. via
<a>), but instead calls a JS function that makes an AJAX request. -
The result is saved to the client-side JSON file
-
The page is re-rendered by joining the static and dynamic HTML (?)
Triggering fetch action on the client
Taking a step back - how does the JS function know what URL to send?
The challenge here is that I assumed that the request will be sent by JS function, so it can act upon it. But so far we've talked about generating plain URLs. So we'd need a way to join the two:
- One way could be to still use plain URLs. But set up some MutationObserver on the client to check for
<a href="....">with fragment URLs forhref. And when it comes across such<a>element, it would register a click (maybe also "enter" keypress) event listener to intercept that.
That way, when the `<a>` is clicked, we could instead run our JS function, knowing the URL.
-
Alternatively we could make it not an
<a>withhref, but it could be any HTML element some custom attribute - This would be practically the same what HTMX does withhx-getI assume. -
The link could be also encapsulated as a template tag, which would internally generate something similar.
E.g. calling
{% link url=url %} ... {% endlink %}would generate<div onclick="djcFetchUrl('/path/to/fragment')"> ... </div>
Generating the fragment-specific URL
The (non-fragment) component-specific URL that I mentioned I implemented will look like this:
/components/ext/url/components/a1b3c4
In theory we could pass in an extra query param like ?fragment=1-2-3
/components/ext/url/components/a1b3c4?fragment=1-2-3
But user may want to use query params to pass data to the component, and mixing the two wouldn't be nice:
/components/ext/url/components/a1b3c4?fragment=1-2-3&fruit=apple
So a better way may be to pass that info as part of the route:
/components/ext/url/fragments/1-2-3/a1b3c4?fruit=apple
Hi @JuroOravec,
In my utilization components do always remain valid HTML, I don't really use the "data-djc-id-" attribute but the concept is the same as your describe, every component has a hopefully unique "id" attribute that we can target as logic requires to reload the component. Say the component is a list with id='X11', we can name an inner component for a modal id="X11-MDL" to better be able to target that modal from within this component, I found this useful to target those components. Django component generated ids aren't always valid html id's, just a heads up.. lost an unreasonable amount of time before I figured that one out. Trying to keep thing sane, if you really need to update an unrelated component other the one that was updated I figure you could trigger a JS event that the unrelated component could listen on thus making a server request to update component contents, HX-TRIGGER can provide a much simpler solution to that problem, or what scenario do you have in mind? You could also make use of OOB.. when I started I made use of OOB but latter abandoned that idea completely as it was creating unnecessary complexity, modifications got harder to make.. instead spent more time on component architecture. For example my object list component is an aggregate of components, the top one is responsible for search, filters, actions and other buttons, the nested is the list itself that also includes pagination controls. Making changes to the filter will trigger a refresh of the whole list and pagination controls, the pagination bar itself is a neste component of the inner list and gets updated with the list itself.
HTMX will get you much further if you break components down.
Best regards,
@ramalhoruben Thanks for the details!
Django component generated ids aren't always valid html id's, just a heads up.. lost an unreasonable amount of time before I figured that one out.
What do you mean by "not valid HTML ids"? As in that because there may be multiple elements with the same ID? Or something else?
[...] HX-TRIGGER can provide a much simpler solution to that problem, or what scenario do you have in mind?
No specific scenario for my case, I was thinking out loud about the approach that @EmilStenstrom mentioned here and here š But agree that I'd leave that up to HTMX.
You could also make use of OOB.. when I started I made use of OOB but latter abandoned that idea completely as it was creating unnecessary complexity, modifications got harder to make.. instead spent more time on component architecture.
Btw what is OOB?
For example my object list component is an aggregate of components, the top one is responsible for search, filters, actions and other buttons, the nested is the list itself that also includes pagination controls. Making changes to the filter will trigger a refresh of the whole list and pagination controls, the pagination bar itself is a neste component of the inner list and gets updated with the list itself.
Btw, @ramalhoruben, ngl, I'd love to see the setup, sounds really interesting! Is your project open source?
@JuroOravec Agree that the approach of working with text diffs doesn't make much sense when we have valid IDs to work with. Happy to scrap that idea :)
I'm going back the the initial ask in this thread. Is there an easy way to both have pages that are backend rendered on the first load, but that then load via JS on consecutive loads, with minimal changes to the backend code?
I wonder if it would be possible to intercept the html that would be rendered if this was a full reload, figure out which components that is in this HTML, and then only update those components dynamically.
Example: The template would have a <a href="/client_list/page/2"> and django-components would intercept the click, fetch that page, see that the this means that data-djc-id-100, data-djc-id-101, data-djc-id-102 needs updates, and only update those components.
I don't know what should happen if the developer wants to update things not managed by django-components, such as a new page layout.
Example: The template would have a
<a href="/client_list/page/2">and django-components would intercept the click, fetch that page, see that the this means that data-djc-id-100, data-djc-id-101, data-djc-id-102 needs updates, and only update those components.
Just clarification - the component IDs are more like GUIDs - when you render the same component / page twice, different IDs will be assigned each time. So we can't make use of that.
However... what DOES stay consistent is the "_class_hash" that the Components have (this hash is based on the path of the source file).
So to elabore further on your idea:
Maybe if we sent to the client a mapping of which component ID was in which position in the render tree, e.g.:
{
"hash": "MyPage_102d34",
"id": "a1b2c3",
"children": [
{
"hash": "Navbar_839fda",
"id": "a1b2c4",
"children": [ ... ],
},
]
}
Then, the second call to /client_list/page/2 would return the HTML, plus the JSON:
{
"hash": "MyPage_102d34",
"id": "b0c2c3",
"children": [
{
"hash": "Navbar_839fda",
"id": "b0c2c4",
"children": [ ... ],
},
]
}
Where:
- The
hashfields remain consistent across the renders. - The
idfields point to thedata-djc-idattributes in the old and new HTMLs.
So with that, we could make the diffing work roughly like so:
- I look at the JSON of my new HTML, and see that at the root we rendered component
MyPage_102d34. - I compare it with JSON of the old HTML, and see that these match.
- If they didn't match, I'd replace the whole subtree with the new content without any futher consideration
- Since they matched, I now compare the HTML of the new and old renders (
outerHTML) of the componentMyPage_102d34 - To get the HTML of the OLD render:
- Construct the component ID as
"data-djc-id-" + entry.id, and getdata-djc-id-a1b2c3 - Search the DOM for selector
[data-djc-id-a1b2c3] - Get the
outerHTMLof the matched elements.
- Construct the component ID as
- Same would be done for the NEW HTML.
- At this point we could compare if the HTML of the component at position
MyPage_102d34changed. - If not, we can leave now.
- If there IS a change, we need to figure out WHERE the change occured - did it happen in CURRENT component (MINUS children), or one of its children?
- So next we would want to get the HTML of the CURRENT component MINUS children.
- For this we could maybe make use of something similar to Elixir's approach - AKA get a list of component's HTML as plain string, delimited by child components:
- Serialize the CURRENT component's DOM subtree into string
- *Somehow* detect any sub-components, extract their IDs, and map the IDs to the child component's hashes
- So I'd end up with a list like `["
hello ", "Navbar_839fda", "
"].- NOTE: Actually we'd want this to be not a list of strings, but of objects, to distinguish between nested components and plain text, so e.g.
[{ text: "<p> hello ", isComp: false }, { text: "Navbar_839fda" isComp: true }, { text: "</p>", isComp: false }].
- NOTE: Actually we'd want this to be not a list of strings, but of objects, to distinguish between nested components and plain text, so e.g.
- Next I would do the same for the new HTML. We might get back the same `["
hello ", "Navbar_839fda", "
"]. - Finally we can compare the new and old CURRENT component MINUS children.
- If these lists were equal, that means that THIS component, MINUS any subcomponents, are the SAME. So the diff occurred in one of the children.
- If these lists are NOT equal, then either 1. Some static content changed, or 2. maybe a component was added, removed, or replaced. But whichever case it is, we'd replace CURRENT component and all of its subtree with the new content.
- Assuming step 13 (that the lists were equal) we'd continue walking down the component tree to find the diffs. Next we'd look for the first subcomponent, which is the
Navbar_839fda. So the overall path could be represented asMyPage_102d34[0] Navbar_839fda[0]or[{ hash: "MyPage_102d34", index: 0 }, { hash: "Navbar_839fda", index: 0 }]. And we'd continue the algo back from step 1).
So defo not easy, but doable.
Here we could use the same trick as for the dependency management, where - if we opted-in - then we'd append a JSON <script> with the component tree structure. And on the client there would be a MutationObserver checking for these JSON scripts, processing them when inserted into the DOM or something.
Since we'd only need to know the component tree structure + add an extra JS script to load (containing the diffing / patching logic), I think this could be implemented as an extension too. In other words, it'd be fairly isolated and independent from the rest of the code.
However, I think I'd make sense to opt into this when rendering the component:
MyTable.render(
...
managed_updates=True, # Or some other name
)
So far we've set extension input only on the Component CLASS, e.g.:
class MyComponent:
class Cache:
enabled = True
But IMO this makes sense to configure on per-render instance - Because I may have two views that render the same component, one where I want to enable this diffing, and second where I don't.
So we should probably add some logic to allow extensions to accept inputs via Component.render() and Component.render_to_response():
MyTable.render(
...
meta={
"diffing": {
enabled: True,
}
}
)
NOTE: The question still remains how the client would intercept, or generally, how it would know that we want to refetch and that particular endpoint upon user clicking on the link.
@JuroOravec Every ID is unique, at least in practice, generation of repeated ID's can theoretically ocurr but that is outside of the scope of this matter.. when I say invalid ID is because the ID's generated might not conform with the Html spec.. to clarify the first character must always be a letter, easy to go around that.. just added a prefix.
OOB is HTMX out of band swaps.. with a single swap you can update multiple components.. if the component that provides the update also provides rendered html to update the other components.. it is possible to do that.. but in my opinion it is not ideal, as it can lead to poor separation of concerns. But might be a valid if approach if you for example have an alert outside of component that you want to fill in upon action result.
@EmilStenstrom If you have components data-djc-id-100, data-djc-id-101, data-djc-id-102 wrapped in a single component then what you are suggesting is possible, working with anchors is/can also be viable without refreshing the whole page if you make use of htmx boost, as I did. I'll give you as an example how I set up my pagination component, they need to provide context tough on how to refresh the list data component (the nested component of the list):
{% component 'objlist_page_i' url_data='{{ url_data }}' page_num='{{ page_num }}'
component_id='{{ component_id }}' text='{{page_num}}'
aria_label='PƔg. {{page_num}}' / %}
class ElementBase(component.Component):
element_type= 'invalid'
template = """
<{{element_type}}
{% for key, value in element_options.items %}
{{ key }}="{{ value }}"
{% endfor %}
>
{% slot "content" default %}{{ text }}{% endslot %}
</{{element_type}}>
"""
def get_context_data(self, **kwargs):
options = kwargs
# Retrieve button caption or set it to '' (default) if not exists
text = options.pop('text', '')
return {
'element_type': self.element_type,
'element_options': options,
'text': text,
}
def get(self, request):
return self.render_to_response()
class AnchorBase(ElementBase):
element_type= 'a'
@component.register("objlist_page_i")
class PageAnchor(AnchorBase):
def get_context_data(self, url_data, page_num, component_id:str, aria_label, **kwargs):
options = {
'href': '#',
'class': 'page-link',
'hx-get': url_data,
'hx-target': f'#{component_id}_body',
'hx-vals': f'{{"p":"{page_num}"}}',
'hx-include': f'#{component_id}_srch,#{component_id}_filt,#{component_id}_ordr',
'hx-swap': "innerHTML show:window:top swap:0.3s",
'aria-label': aria_label
}
full_options = kwargs | options
return super().get_context_data(** full_options)
Every ID is unique, at least in practice, generation of repeated ID's can theoretically ocurr but that is outside of the scope of this matter.. when I say invalid ID is because the ID's generated might not conform with the Html spec.. to clarify the first character must always be a letter, easy to go around that.. just added a prefix.
@ramalhoruben Thanks for explaining! In that case I think the IDs don't necessarily need to change, as other systems could potentially have different constraints, and it's not the role of the IDs to confirm to that. But I think we can at least mention that in the docs - I've made a ticket for that here https://github.com/django-components/django-components/issues/1084
OOB is HTMX out of band swaps.. with a single swap you can update multiple components
That's actually pretty cool, didn't know about it! š (link for context).
For Emil's idea, though, I'd go with the approach in my last comment. If I understand the OOB correctly, we'd need to prepare the diff on the server. To do it on the server, we'd need to either send the UI state to the server, or store it there from the get-go. And then we'd be again in the Tetra territory, as I mentioned here, and IMO outside of the scope of django-components.
If you have components data-djc-id-100, data-djc-id-101, data-djc-id-102 wrapped in a single component then what you are suggesting is possible, working with anchors is/can also be viable without refreshing the whole page if you make use of htmx boost, as I did.
Great idea with the HTMX Boost, again another thing I didn't know.
Yeah, we could do something like that for this feature.
So I imagine that the flow to detect when to trigger refetch and differential update (or whatever is a proper name for this technique):
- When user marks some rendered component as using these differential updates, we append a JS script that will manage it.
- The rendered component will also have appended a
<script type="application/json">tag that contains the component tree structure of the rendered component.
-
NOTE: We'd also need user to define some unique ID to identify this rendered component when we then fetch it the next time.
def my_table_view(request): page_id = request.GET["id"] return MyTable.render_to_response( ... meta={ "diff": { "enabled": True, "id": "my_table__" + page_id, } } )The upside of defining the ID like this is that it would then be independent from the endpoint. So in theory different endpoints could make updates to the same component in the page.
-
The JS script that we've added in step 1. will detect on initial page load the
<script type="application/json" data-djc-diff>in the DOM, and will be able to associate that together 1) the component tree from the JSON, 2) the HTML element(s) for the component in the DOM, and 3) the component diffing ID"my_table__7". -
The JS script that we've added in step 1. will also override the
<a>and<form>elements on the page, doing a same thing as HTMX Boost.-
NOTE: Here we'd need some way to configure which
<a>and<form>tags to apply to. I can imagine two defaults -- Either apply our HTMX Boost only to
<a>/<form>INSIDE the rendered component (preferable) - Or apply it to ALL
<a>/<form>, even outside the rendered component.
And to give users flexibility, they could specify a selector to define which elements to target. Since the selector string can contain commas to define several selectors, a string would be sufficient
def my_table_view(request): page_id = request.GET["id"] return MyTable.render_to_response( ... meta={ "diff": { "enabled": True, "id": "my_table__" + page_id, "link_selector": "a:not(.hidden),form:not(.hidden)", } } ) - Either apply our HTMX Boost only to
-
NOTE: The input to the
metacould be alternatively set on a component class:class MyTableFragment(MyTable): class Diff: enabled = True link_selector: "a:not(.hidden),form:not(.hidden)" def get_id(self, request): page_id = request.GET["id"] return "my_table__" + page_id def my_table_view(request): return MyTableFragment.render_to_response(request=request)
-
-
As last step, when the end user clicks on any of the targetted links, we make an AJAX request. And if the response contains
<script type="application/json" data-djc-diff="my_table__7">as one of the root elements, then we'll know that the HTTP response is trying to make patch updates to the component identified by
"my_table__7".-
If there's no such match for
my_table__7, then we either raise error or fall back to the default behaviour of the<a>/<form>elements. -
Likewise, if the HTTP response doesn't contain any
<script data-djc-diff>tags, we should fall back to the default behaviours of the elements.
-
So to recap, I think this is the minimal interface for enabling users the diff-based updates to their UI:
class MyTableFragment(MyTable):
class Diff:
enabled = True
link_selector: "a:not(.hidden),form:not(.hidden)"
def get_id(self, request):
page_id = request.GET["id"]
return "my_table__" + page_id
def my_table_view(request):
return MyTableFragment.render_to_response(request=request)
@JuroOravec About your reply to my example code, I don't understand how this has to be 15 steps, and include JSON. This is the flow I'm imagining:
- Client: Intercept all link clicks and remake them as AJAX calls to the same URL
- Server: Intercept the call and return just a list of which components are part of rendering the template in that response
- Client: Parse the response to find all components
- Client: Make calls to update all those components and return the HTML for those fragments
- Server: Return the HTML for each component
- Client: Update those components if they are different from the current HTML, probably using virtualdomCan you tell me why this wouldn't work?
Advantage is that this needs no changes to the backend code for the developer, everything is handled by djc.
I don't understand how this has to be 15 steps, and include JSON
Because my approach:
-
Would work within bigger structure - e.g. maybe I want only a certain part of the page to be managed by django-components like this.
-
Would also handle dynamic number of components. (e.g. page has a list with 3 items. 4th item is added on another page. I reload my first page, I would now expect to see 4 items). If I'm reading your flow correctly, your approach wouldn't allow that.
-
Your approach doesn't handle the case when components contain nested / child components, and inputs are passed from parent to child. In your steps 4 and 5 you assume that we know all inputs to all rendered components, and that these inputs can be exposed to the client.
For this one, consider this - You cannot simply request to render the
Childcomponent from the client, because the logic that decides what thethemewill be is defined inMyPage:theme_light = { "button": "bg-green radius-2 border-solid", ... } theme_dark = { ... } class MyPage(Component): template = """ <div> {% component "child" theme=theme / %} </div> """ def get_context_data(self, dark: bool): theme = theme_dark if dark else theme_light return { "theme": theme, } class Child(Component): template = """ <button class="{{ theme.button }}"> Click me! </button> """ def get_context_data(self, theme): return { "theme": theme, }
@EmilStenstrom mostly in tune with my toughs, keeping it simple.
@JuroOravec if you picture the components and their relationship's as a tree structure then a refresh of the root node will as a side effect ask to re-render the whole tree, you won't have to keep a list of which components to update, they are implied, which I find convenient. If you have multiple roots or subtrees you intend to update at the same time with a single action then you can make use of hx-triggers, or hx-swap-oob as I suggested before. If you intend to update only the root node and only the root node while keeping the nested components unchanged then the current approach can't... but instead I can refactor my structure such that the component is a sibling of the component and not within, because when I nest a component within another I assume this relationship and constrain. If I understand the example correctly I think you will always need to pass the theme information to the "view component", yes, both to root node or to any of it's childs if you happen to request child updates.. I would suggest you make the theme information a field on your page, a checkbox perhaps, and use the hx-include directive to provide that bit of context to either component when requesting a refresh. If you think my example and the pagination button I provided before then you can see I also had the same problem, when navigation through list I need to provide context to list data such that filters and sorting information remains consistent between different pages, there's no other way around, at least no sane solution that I am aware of. Application performance is clearly not a big concern for me, as I am much more concerned about doing more with less time, being more productive, that's why I picked Django after all. But still... I would have some concerns about that "diffing". For simple calculations I would expect a full re-render of a large chunk of the page to be cheaper than calculating and providing updates to specific components only. But don't take my word on it.. I am wrong a lot of times. š