Cache-bust dynamic configuration files and theme CSS

[ybnd]

References

• Fixes https://github.com/DSpace/dspace-angular/issues/3992
• Fixes https://github.com/DSpace/dspace-angular/issues/1961

Description

DSpace Angular's production server can modify the configuration it serves to CSR-only clients through YAML or environment variables. However, these files can remain cached in the browser and leave users with out-of-date configuration until the TTL runs out or the user does a "hard refresh". On build time this sort of problem is solved by saving the content hash as part of the path of each file (JS, CSS, ...)

We introduce HashedFileMapping to bridge the same gap for dynamic content generated after the server is built:

• Files added to this mapping on the server are copied to a hashed path
• A copy is injected into index.html, where clients can read it out and resolve the hashed paths
• If a given path is not found in the mapping, the client will fall back to the original version (to prevent errors in development mode)

With this mechanism we can ensure updates to config.json and similar files take immediate effect without losing the performance benefit of client-side caching.

Compression support is included in anticipation of a pending PR for https://github.com/DSpace/dspace-angular/issues/3961, where the initial size of config.json can become large enough where compressing it would be a plus.

Discussion

• It's not clear how/if we can unit test this at the time of writing
• Should we inlude a configuration flag to toggle this feature on/off? We may want to roll it out as opt-in so downstream users can wait it out until it has been battle-tested a bit.

Instructions for Reviewers

To test:

1. Run DSpace Angular locally, in production mode
2. Confirm that you see the following files:
   • [dist]/assets/config.<hash>.json → hashed configuration file
   • [dist]/dspace-theme.<hash>.css → hashed theme CSS file
   • [dist]/index.<hash>.html → index file with an extra script#hashed-file-mapping element containing the mapping of plain paths to hashed paths
3. Confirm that config.<hash>.json and dspace-theme.<hash>.css are correctly served to the browser, and that the app runs/looks as before the PR
4. Confirm that https://github.com/DSpace/dspace-angular/issues/3992 is resolved

Checklist

This checklist provides a reminder of what we are going to look for when reviewing your PR. You do not need to complete this checklist prior creating your PR (draft PRs are always welcome). However, reviewers may request that you complete any actions in this list if you have not done so. If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

• [x] My PR is created against the main branch of code (unless it is a backport or is fixing an issue specific to an older branch).
• [x] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & specs/tests), or I have provided reasons as to why that's not possible.
• [x] My PR passes ESLint validation using npm run lint
• [x] My PR doesn't introduce circular dependencies (verified via npm run check-circ-deps)
• [x] My PR includes TypeDoc comments for all new (or modified) public methods and classes. It also includes TypeDoc for large or complex private methods.
• [x] My PR passes all specs/tests and includes new/updated specs or tests based on the Code Testing Guide.
• [x] My PR aligns with Accessibility guidelines if it makes changes to the user interface.
• [x] My PR uses i18n (internationalization) keys instead of hardcoded English text, to allow for translations.
• [x] My PR includes details on how to test it. I've provided clear instructions to reviewers on how to successfully test this fix or feature.
• [x] If my PR includes new libraries/dependencies (in package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
• [x] If my PR includes new features or configurations, I've provided basic technical documentation in the PR itself.
• [x] If my PR fixes an issue ticket, I've linked them together.

[github-actions[bot]]

Hi @ybnd, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[ybnd]

@tdonohue adjusted the labels since this PR was opened against 7.x

[ybnd]

Note: when running multiple DSpace Angular instances in PM2 cluster mode this would cause the same filesystem operations in parallel, could cause problems.

Could be addressed with Redis similar to https://github.com/DSpace/dspace-angular/issues/3110#issuecomment-2188477157

[ybnd]

Update: while debugging #4081 I realised this PR also gives us the opportunity to prefetch assets.

Angular does similar for global CSS, but doesn't know about our themes or JSON configuration. If we're already making changes to the index HTML it's easy to include the appropriate links

Before:

After:

[kshepherd]

@ybnd could you rebase + repush to force the build tasks to re-run ? the unit test was so long ago that the logs aren't available so it's not clear why it failed

[ybnd]

@kshepherd if possible I'd like avoid rebasing to keep the history consistent with the "clean 7.6" version of the branch (already using that elsewhere)

Looking back now I did find a few small improvements to make (mainly with the theme prefetching part), so that will revive the CI just as well.

[github-actions[bot]]

Hi @ybnd, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[ybnd]

@MW3000

There is no index.<hash>.html file generated, just the normal index.html file! But I guess, that is just an error in this description? As I understand the code, there should not be an index.<hash>.html file, as the normal index.html is being excluded from cashes?

Good eye, that bit stayed behind in the PR description from an earlier iteration.

Initially I tried to use the same hashing mechanism for the index file, but that lead to tricky edge cases depending on how exactly it was requested → better to rely on Cache-Control headers

Amended the description to reflect this ✔️

[tdonohue]

@ybnd : I took a quick look at the code in this PR today. Overall, it looks useful. But, I'm wondering why we are starting with the dspace-7_x branch instead of testing this on main? Do we have a stable version of this PR against main that we can also review?

I'm not against also potentially backporting this all the way to 7.x. I just often find that forward ports are more difficult (because of new lint rules and similar), so I think it's important to get a main version of this PR ready in order to move this change forward.