operations
operations copied to clipboard
openstreetmap
Consider serving static content from a CDN
According to awstats, one of the most popular URL on osm.org is /export/embed.html, being 170% more popular than openstreetmap.org/ itself.
If we serve this from the static CDN, we'll be able to deliver it faster for all the casual OpenStreetMap consumers, unloading main website.
We (that is to say @Firefishy and I) have discussed this before - not for this page specifically, but for the static assets in general. I'm actually more interested in helping the main page load quickly then I am in helping third party sites load the embed page which isn't our main mission.
We have discussed using the existing tile CDN for this, though I wonder if the different access patterns and the number of tile requests it servers would mean it wouldn't do as well as it could for these assets.
We'd see considerable gains by reducing the TTFB of static assets, which the CDN would help with.
Taking the start time from dns lookup of www.openstreetmap.org, it's 2.8s to first tile. Removing the TTFB from static assets takes out 300ms, and boosting the download speed of application.js could take out up to 1000ms of additional time. Both of these times are significant and shown to boost various user metrics. Because I'm on the west coast of Canada, this is about a best case for adding a CDN.
It didn't limit load time because application.js had to download, but the time spent queued waiting for a socket was significant. HTTP/2 would help this.
boosting the download speed of application.js could take out up to 1000ms of additional time
Today, application.js is still the most expensive part to load, and to a large extent includes i18n texts for all 95 supported languages. I don't know how easy it is in Rails to reorganize this content in a way that only relevant parts are downloaded.
The whole point is not to break it up so that it can be downloaded once and used for all the pages and so that there is only one connection setup latency.
You can do whatever you want but the general theory is to create large bundles rather than downloading lots of little fragments.
Well, as a user I’m typically interested in texts for my own language, still I need to download all other 94 languages as well. So the idea would be to serve a very small application.js (possibly with relevant texts only?) along with a language bundle for my language.
Separating out the locale texts is a reasonably logical idea though I think it might be quite hard to actually do.
You make it sounds like locale texts are the main purpose of application.js though, which they definitely aren't. They may make up most of the volume (I haven't checked) but they're certainly not the main purpose.
Incidentally we do only serve up relevant texts - there is a filter which selects only certain texts that are needed on the client side to go in the js.
They may make up most of the volume (I haven't checked)
This seems correct, a quick check showed about 90% for translations vs. 10% other application logic in that file, both compressed and uncompressed.
As best I can tell i18n-js can't do per-locale files when working via the asset pipeline as our configuration currently does.
I think it could be done by operating in export mode instead, which would be find in production but a pain in development so we would likely need some hybrid model.
Alternatively we would need to try and extend i18n-js to support per-locale files in asset pipeline mode.
@tomhughes : could we close this? The issue was resolved in https://github.com/openstreetmap/openstreetmap-website/issues/1949
Right. CDN was suggested as one option to speed up loading, but this didn’t consider the root cause of the issue: massive static file size increase by always including all translations. This has been fixed in the linked issue.
IIRC those savings were quite significant with more than 10TB / month.
We think the priority for this is the community.osm.org site, as it involves many more files that would benefit from the CDN
https://community.openstreetmap.org/ has been using fastly since 20th November 2022.
From the fastly side it would be very easy to setup for www.osm.org... but
I don't think it would really be worth it anymore. The website only uses around 234KB of network transfers to load (excluding CDN tiles and current bloated banner) with assets mostly Brotli compressed or optimised images. Modern Chrome download it all via a single pipelined connection. The copyright page as a subsequent page only uses ~16KB transferred.
I propose we close this ticket. We can always re-open or create a new focused ticket if the need arises.
The website only uses around 234KB of network transfers to load (excluding CDN tiles and current bloated banner)
The user-facing impact of a CDN is to reduce TTFB, not transfer time. In 2016 there were 8-10 TTFBs before map tiles were loaded
These days, it's much better thanks to HTTP/2
There's 1-2 TTFBs before DOM content loaded (blue line), so we might be able to shave 10% off of the load time.
There's 1-2 TTFBs before DOM content loaded (blue line), so we might be able to shave 10% off of the load time.
Would a 10% potential improvement be enough justification for adding a CDN? Additional components add risk and complexity.
Would a 10% potential improvement be enough justification for adding a CDN?
Most websites would do it. The potential issue I see is that because it'd be on another hostname, there would be another DNS and connection establishing delay. I guess that's not a serious problem with 0-RTT, which fastly supports.
Additional components add risk and complexity.
Because the files we'd serve by CDN are static, I think the risk and complexity is minimal on the operations side, if Rails allows easily pointing to a different domain name for that content.
Yes telling rails to point assets at a different hostname is easy.
Synchronising the updates of all the web servers and the CDN not so much.