beeware.github.io
beeware.github.io copied to clipboard
beeware
Replace Netlify
It appears we're outgrown Netlify.
We are currently using Netlify to provide PR previews of pull requests. However, Netlify has recently made some radical changes to their pricing structure, reducing the number of build minutes allowed per month on the free tier, and reducing the time limit on builds. We hit the first limit on the 3rd of October; we hit the second limit on every build in October.
If we were using Netlify for main site deployment, I'd throw money at the problem and make it go away; but this is for PR previews. There has to be a better (and cheaper) way.
To Reproduce Steps to reproduce the behavior:
- Go to any recent PR, and look at the CI actions
Expected behavior
A PR build should be visible
Additional context
We already have an AWS account and pay for S3 storage, so the best solution I can think of would be to have https://pr123.ci.beeware.org redirect to a configuration serving s3://beeware-website/ci/pr123 as the root document, and publish to that bucket at the end of a CI run validating the build. Copying the files to a bucket is trivial; the only complex part is the redirection (and making that work dynamically, so we don't need to register a subdomain for every PR). I suspect Cloudfront will be involved.
This may be about to become critical - Netlify has started sending me emails talking about how my sites will be suspended if I don't give them a credit card.
@dzc0d3r We already use Github Actions for CI, so yes - I imagine Github Actions will be used.
If you're referring a specific action (such as the the two you mention) that deploy to GitHub Pages - that might be a possible solution. However, from first inspection:
- pr-preview deploys to
https://[owner].github.io/[repo]/pr-preview/pr-[number]/; that means the root URL of the generated site isn't/; that will break any absolute URL references in the generated site - or worse, will fall back to resources on the main beeware.org domain, hiding any problems caused by resource changes. The subdomain-based URL scheme I mentioned was deliberate. - surge-pr-preview is a hook into a commercial service (surge.sh) that offers a "free tier". Given that we've just been burned by a provider changing the rules for a free tier, I'm not rushing to move to another free tier.
@freakboy3742 My bad i didn't pay attention to root URL for the pr-preview and yes using surge-pr-preview isn't a solution since they can change rules for their free tier
Publishing to the bucket at the end of a CI isn't the main problem .. But like you said the complex thing is making https://pr123.ci.beeware.org redirect to a configuration serving at s3://beeware-website/ci/pr123 as the root document (dynamically for each PR) .. by any means my AWS knowledge isn't that good so i can't think of anything so far .
I was thinking why not to use beeware.org/preview/pr-number but since we're using absolute path because of Atom feed that may break things, although using HTML Base tag <base href="https://www.beeware.org/"> may work
@dzc0d3r The base tag in RSS is only one example of a potential problem. Consider the image on the homepage: <img href="/static/images/brutus-270.png">. That's an absolute URL. If you mount the PR-preview site at https://beeware.org/preview/pr-number, the img will be pointing an image on the main site, not the one on the PR preview site. The same will be true for every image, every hyperlink, and so on.
@freakboy3742 Yes, using https://beeware.org/preview/pr-number isn't a good idea in general.
For using subdomains for PR previews like https://pr123.ci.beeware.org I think i found something that may help using AWS Amplify AWS amplify docs but it requires Amplify and Route 53
Other solutions seems more complicated and will require using CloudFront, AWS CDK (and write some code), S3 Bucket, Github actions, and the preview link will be at gibberish.cloudfront.net and Route 53 for subdomains on a custom domain like https://pr123.ci.beeware.org
@dzc0d3r From an initial look, Amplify might be a good option - certainly an easier option than a custom CloudFront+ configuration.
I've setup a preview of firebase hosting previews on my project fork
https://github.com/glasnt/beeware.github.io/pull/1
I've also already setup a $1 billing alert on that account, just in case any costs are incurred.
Could this be used eventually https://github.com/marketplace/actions/deploy-pr-preview ? (v2 will support forks as well)
@goanpeca Not easily - aside from the fork, the sticking point is that all the PR-preview actions that I found assume you'll be deploying to a /pr/1234 subfolder (or similar). This means that all the URLs on your site need to be computed relative; a link to the homepage needs to be rendered as /pr/1234/ in the preview site. It may be possible to do this, but it will always be fragile because all it will take is a single hard-coded URL to a resource and your preview site won't be representative any more.
What we need is a preview to a subdomain (e.g., pr-1234.docs.beeware.org), so that URLs don't need this kind of root re-writing.