HashedFilesMixin + GoogleCloudStorage break

[ewjoachim]

trafficstars

When defining a storage like this:

class ManifestGoogleCloudFilesStorage(
    storage.HashedFilesMixin,
    gcloud.GoogleCloudStorage
):
	pass

We would imagine that it plays nicely together.

It may be the case, but it may be not, depending on the settings. Actually, there are 2 problems, both of which are solved by setting GS_QUERYSTRING_AUTH to False. That might seem logical if you have a deep understanding of hashed storages & google URLs, and which can make you spend some good hours debugging.

Note that if GS_QUERYSTRING_AUTH is set to True (default), your URL will have a query parameter ?signature=<big blob of base64 containing slashes changing everytime>.

Django's HashedFilesMixin breaks on / in query strings

When post-processing the files, django makes a replacement where it assumes that / are only found in the path, not in the query string. This means that for the the URLs in CSS files, instead of being changed from url("foo.css") to url("foo.12345abcdef.css") will be changed to url("Ef35dT==") (the last part of the signature base64 after the last /). So it breaks.

Django's HashedFilesMixin only works if computing the URL twice on the same file creates the same URL

The signature will change each time (I believe it contains a reference to the generation timestamp so that the URL is not valid indefinitely).

The way the HashedFilesMixin works is that it will keep post-processing the files until it detects that post-processing them produces no changes compared to the previous path. Because the URLs change each time, this means Django is stuck in a loop. After it reaches the maximum number or tries (storage.max_post_process_passes), it raises.

Conclusion

GoogleCloudStorage seems to be unusable unless if GS_QUERYSTRING_AUTH is set to False. This should be prominently documented at least, and maybe there should be a runtime check.

Just to be extra explicit, GS_QUERYSTRING_AUTH=False solves everything and it works nicely.