Use server metadata to determine if document changed
To optimize http requests, it would be good to send If-Modified-Since and/or If-None-Match headers and use ETag and Last-Modified.
I'm thinking it might make sense to put these into a database in production?
Relying on the server through a HEAD request to tell us whether the document changed instead of downloading it is indeed a good way to decrease the load… but also means we trust the server :wink:
Since we have performance issues at the moment, I'm in favor of postponing this optimisation.
When it is considered, we should in any case first assess the reliability of the method by comparing the results of HEAD information with the actual diff over a couple of days 🙂
Following https://github.com/OpenTermsArchive/docs/pull/43, we can refuse this suggestion in application of Principle 1: Never trust the services.