htmx
htmx copied to clipboard
bigskysoftware
Preload extension preloads URLs but navigating there loads the data again
Hi there! I might be confused on how the preload extension is supposed to work. See this screenshot of the chrome dev tools inspecting the network activity when navigating htmx.org:
Starting from the front page I hovered over the docs link in the main navigation. preload.js loaded the URL, as expected. Then I clicked on docs and now htmx.js loaded the URL again from the server, seemingly without using the preloaded data. As soon as the docs page had been rendered, moving the cursor just a little bit (it was still resting on the docs link), triggered preload.js again, resulting in loading the URL again from the server. Same scenario with the reference link. Three server requests instead of one for each link with preload.
Am I missing something here? I would have expected that after preloading a URL, navigating towards it would not need another server request.
P.s.: I don't think the error in the console is related, it just points out a 404 for https://unpkg.com/[email protected]/components/prism-ecmascript.min.js:
Hey, thanks for posting this. I'll try to look into it soon. Without doing research, my first thought is that there might be different headers between the two requests, which is why your browser is not using the preloaded/cached content. Do you have anything in your code that changes the request headers?
Hi @benpate , actually I would have expected that HTMX would not send a new request to the server at all, when an URL was preloaded. That's how I usually do it in my setups.
For this extension, we're are leaning on the browser's built-in caching mechanism. It does not store a separate copy of the server response, which could get cumbersome really quickly. How do you usually do this, then? Do you have sample code that you could share? If there's a better way, then we should do that :)
I guess this means that other server headers could also be messing with the browser's cache -- for instance, a no-cache header would definitely prevent it from working.
Sorry for my late reply. I understand. If this is out of scope for HTMX, I will continue using my own implementation.
I am using a Map (caniuse) for that kind of functionality, with the URLs as keys and the document content, saved scroll positions etc. as data. I also have a cleanup mechanism running, so that I never have more than a certain amount of cached documents in this map (to prevent RAM issues). Here is the structure of my cache object, maybe this helps if you ever want to implement something like this in the future:
0: {
key: "https://mysite.test/"
value: {
data:
$document: "<!DOCTYPE html> ...",
openAccordeonItem: "",
scrollPositions: {root: 0},
url: "https://mysite.test/",
},
keepInCache: true,
state: "loaded"
}
}
1: {"https://mysite.test/news/" => {...}}
2: {"https://mysite.test/about/" => {...}}
3: {"https://mysite.test/members/" => {...}}
4: {"https://mysite.test/rooms/" => {...}}
5: {"https://mysite.test/more-information/" => {...}}
6: {"https://mysite.test/contact/" => {...}}
EDIT: The above object would be the state after navigating 7 links in my site.
Hey @hirasso - I have seen some odd examples where the preload extension's HTTP headers don't match those in the final request, leading to duplicate requests. I still want to track those down, along with updating the extension with some of the lessons I've learned from using it. But, I DO want to stick with using the Browser's built-in cache because browser vendors have put so much work into optimizing these tools, and I want to use the platform as much as possible.
But htmx is immensely flexible, and you could certainly make an alternative extension tha stores content in a Map of some kind. Absolutely feel free to use the existing extension as a baseline for migrating your own tool into an extension. :)
Sorry for my late reply. Got caught-up with live ;)
Without doing research, my first thought is that there might be different headers between the two requests, which is why your browser is not using the preloaded/cached content. Do you have anything in your code that changes the request headers?
Today I started investigating this again.
The request headers for the preload request:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-DE;q=0.8,en;q=0.7,en-US;q=0.6
Cache-Control: no-cache
Connection: keep-alive
Host: htmx.test
Pragma: no-cache
Referer: https://htmx.test/
sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36
The request headers if I actually click the link:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-DE;q=0.8,en;q=0.7,en-US;q=0.6
Cache-Control: no-cache
Connection: keep-alive
Host: htmx.test
HX-Boosted: true
HX-Current-URL: https://htmx.test/
HX-Request: true
HX-Target: container
Pragma: no-cache
Referer: https://htmx.test/
sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36
There actually is a difference, and it's the HX-* headers for the actual link click!
HX-Boosted: true
HX-Current-URL: https://htmx.test/
HX-Request: true
HX-Target: container
I've also been experiencing this with the latest version of Chrome on Windows 10 so I would say chances are good that this affects a large number of users. My findings are the same as @hirasso back in August regarding the headers. Having the preload extension working properly would be a huge boon for me!
Absolutely. And, sorry I haven't been able to make much headway on this recently.
The short answer is that it's probably something with the headers being sent -- the extension tries to mimic a regular HTMX request, but there must be differences.
Could you open up the HTTP requests for a typical transaction and pull the headers for a) the preload request and b) the actual request? What headers are different? In hiraso's example above, it was HX-Boosted, HX-Current-URL, HX-Request, and HX-Target. If this is your experience as well, then we can try to improve the "mimicry" that the preload extension does.
One other hack is to try using the Vary header to limit the specific headers that your browser uses to bust the cache. This is a whole science in its own, but for this purpose, if you added a header that only specified Vary:Cookie then your browser should not (in theory) have to reload the URL just because these other headers are different.
Hi, i'm having the same issues with using the preload extension. Resources are being downloaded twice. The issue seems to be a difference in these headers : hx-target: and hx-trigger.
When id and name are set on the element these differ e.g one one request they are hx-target: root hx-trigger: root
And on the other they match those on the element eg hx-target: tst hx-trigger: tst
When no id and name are set on the el, the headers are only present on one of the requests, and are set then as hx-target: root hx-trigger: root
Thanks for this report. Long-term, I think we'll need another way of handling preloads. For now, does the Vary header work for you?
Sorry you're having trouble, @onemoreahmad. Can you provide more information? Have you tried working with the Vary header?
Hi @benpate, thanks for your reply. I really don't know how this supposed to work, but I did try few things.
1 instantclick.io
I've tried few similar tools to see how it works, first, I checked http://instantclick.io/
The way it works is once I hover to the link, it fires a get request to the page, and when I actually click on the link, it opens fast without issuing anything in the network tab in the dev tool.
Eventhough maybe instantclick doesn't use ajax between pages, just a regular links.
2 quicklink
Then I checked quicklink, in there website when I open network tools, nothings shows up when I hover or click on links, but I tried it with my project which I use HTMX. With a simple installation
<script src="https://cdnjs.cloudflare.com/ajax/libs/quicklink/2.3.0/quicklink.umd.js"></script>
<script>
window.addEventListener('load', () => {
quicklink.listen();
});
</script>
It work on my website only on the current links exists on the page when it first loaded, but next time I click on the link again, it just work normally with a full server request.
3 using HTMX Preload extension
when I install the plugin, when I hover the link, it gets loaded from the server, and then when I click on it it loads it again, no cache nothing.
When I check headers requests both request it identical, the only difference is the xsrf token, and both has Vary key
Hey.. sorry I couldn't dig into this until now. It's hard to see all the header information on your screenshot, but if your responses have Vary:Accept-Encoding then it's worth it to check the Accept-Encoding header on your requests. They might be different.
Working with the browser's cache IS a little fiddly, and is outside of our control with this extension. I'm hoping that we get more access to the internals in htmx 2.0, which might give us more control over how this caching works.
In the meantime, I'm using a more aggressive version of this code that basically just loads the content on mousedown (without waiting for the mouseup or click events). You're welcome to use this if it makes sense in your app.
@onemoreahmad I'm noticing cache-control: no-cache in headers from your server... EDIT: and for OP.
When I hit this, I added an appropriate max-age to my server responses to tell the browser that it's indeed ok to cache requests. Works great..
I am having this issue when combining hx-boost and preload.
From documentation:
<a href="/server/1" preload>WILL BE requested using a standard XMLHttpRequest() and default options (below)</a>
<button hx-get="/server/2" preload>WILL BE requested with additional htmx headers.</button>
The issue combining hx-boost and preload:
<div hx-boost>
<a href="/server/1" preload>DOUBLE REQUEST. Preload will not send HX-Request, but the click will</a>
</div>
Here HTMX will make 1 request with HX-Request header and one without it. The response contains Vary: HX-Request, so the browser knows both requests are different and that the response for the preload request can't be used as cache for the following request.
Removing Vary: HX-Request is not an option because the browser needs to know both responses are actually different, otherwise a full refresh could result in it showing only a partial response instead of a full page.
I think the right thing to do would be for this extension to check if the link is inside a hx-boost context.
Hey @yokomizor - I'm sorry you're having trouble with the extension. I don't have much experience using hx-boost but it looks like there's an HX-Boosted request header that is also sent to the server. Is this a part of your Vary header? Could it work to resolve the "double request" problem?
Hey @benpate,
It's all good. The trouble is actually very minor and very easy to patch manually from my side.
About your suggestion, let me try to explain why this is not what I was looking for.
Take a look at this piece: https://github.com/bigskysoftware/htmx/blob/master/src/ext/preload.js#L45-L69
// Special handling for HX-GET - use built-in htmx.ajax function
// so that headers match other htmx requests, then set
// node.preloadState = TRUE so that requests are not duplicated
// in the future
var hxGet = node.getAttribute("hx-get") || node.getAttribute("data-hx-get")
if (hxGet) {
htmx.ajax("GET", hxGet, {
source: node,
handler:function(elt, info) {
done(info.xhr.responseText);
}
});
return;
}
// Otherwise, perform a standard xhr request, then set
// node.preloadState = TRUE so that requests are not duplicated
// in the future.
if (node.getAttribute("href")) {
var r = new XMLHttpRequest();
r.open("GET", node.getAttribute("href"));
r.onload = function() {done(r.responseText);};
r.send();
return;
}
Here we see that there are 2 different strategies for requesting content for preloading. For <a href=...> we will not send HX-* headers. It would not matter if Vary contains only HX-Boost or only HX-Request.
This is actually expected behavior according to preload extension docs. It seems reasonable to do this, since usually users clicking a <a href=...> will not trigger a AJAX request.
However, using hx-boost makes it so that <a href=...> requests will go via AJAX, and that the server can return only a partial, instead of the whole page. The server knows it has to return a partial based on these HX-Request and/or HX-Boosted headers.
The issue is that preload is not aware of hx-boost. So it asks for the server for a full page (by not adding HX-* headers), which ultimately preloads a request that will not be used, since boost will make a AJAX call, and not a "full reload".
Does that make more sense?
It's all good. The trouble is actually very minor and very easy to patch manually from my side
Hey @yokomizor, I'm running into the same issue of combining hx-boost with the preload extension. May I ask how you decided to solve the problem?
