normandy
normandy copied to clipboard
mozilla
Support configurable domain for addon urls
Currently, I believe normandy uses the value of DJANGO_AWS_STORAGE_BUCKET_NAME when generating the URL to return for a recipe's addonUrl.
We want to be able to put a load balancer or CDN in front of the bucket. This means we need to support configuring an alternate value to use in the URL instead.
Are we planning to do this only in AWS, only in GCP, or both? Our storage backend doesn't have any shared settings between the two, so I'll have to configure them both separately. If we are only going to do this in one environment or the other, I'll skip the ones we don't need.
We could do it on AWS, but I don't think it's worth the effort given our migration timeline. Just adding support for this when running on GCP sounds fine.
Ok, so more specific questions then. Our Storage backend doesn't support custom domains for GCP (it does for AWS). Can you give an example of what a URL would look like in Cloud Storage, and how that would map to a CDN URL?
That actually makes sense to me, since the domain we're accessing it at is independent of the storage mechanism. I guess the storage library does have that option for AWS since you can assign a custom domain name directly to an S3 bucket.
I created a bucket named bpitts, a folder named folder in it, and a file named file in that.
Direct to bucket: https://storage.googleapis.com/bpitts/folder/file Via CDN: http://35.201.107.250/folder/file
I'm using the IP address in that example, but in the real world we'd use a domain name.
The difference is that when accessing GCS directly the bucket name is part of the URL, but when we go through the Load Balancer+CDN it is not.
The fact that it strips the bucket name makes this a bit more awkward. If only the hostname need to change, it would be much simpler. I'll play around with this and see if I can get something sensical.
If it simplifies your implementation a great deal, we could configure the load balancer to recognize the same path as storage.googleapis.com.
Basically, at the load balancer you get to map a path to a bucket. In the example I had just made the default backend the bpitts bucket. I could also map the path /bpitts/ to the bucket.
I think that would work well. The alternative I have in mind is to recognize things that start with https://storage.googleapis.com/{bucketName}, and do a string replace on that with the path. That gets really backend specific, and I'd prefer to be more neutral. Simply asserting that the add-ons should be on a specific domain is much simpler, and more generic.
FYI we should be ready on the GCP side to test this functionality this week.
So after trying to set it up it turns out that what I said earlier was wrong, it is not possible for me to have a path in a url map route to the root of a bucket. I can route a path to the bucket, but that path is not stripped from the request to the bucket. E.G. Normandy would upload https://storage.googleapis.com/bucketname/file and tell clients to download it from https://configureddomain/bucketname/file . The load balancer would route that download to https://storage.googleapis.com/bucketname/bucketname/file, which won't work due to the duplication of the bucketname.
So I think your idea of finding storage.googleapis.com/{bucketName} and replacing that string with the configured domain is the way to go.
I think this is a high priority now. I'd like to put Cloudfront in front of the GCS addons bucket.
several issues are related to add-ons, gcp, and other our future direction... batch these together to talk at once with the add-on work and ops.
this depended on a bunch of client work that happened. the priority is lower now - also add-on manager work is likely happening that will remove this situation. so we will likely grow out of this issue - though configurable and backwards compatible is a harder to do than fixing for forward.
Future: action arguments being defined in model arguments may be a direction we head (over json blobs).
