json-ld
Convert to Eleventy.
Status:
- Second try at converting to a static site generator.
- First try was to Jekyll in 2017: https://github.com/json-ld/json-ld.org/pull/461.
- Much of the discussion there applies here after many years.
- There are many many many SSG choices. This conversion is using Eleventy. If the community wants to use another system, it's likely these conversions are a good start with similar patterns.
- What's happening in the initial commits here:
- Handles pass through of all useful (and more) files that were served before.
- Converts the runtime php indexing code to build time js code.
- Some tweaks for uniformity.
- The plain apache+php hosting we use now could probably host this output with a few operational changes.
- Templates and various upgrades and improvements are not yet done. Solving issues one step at a time.
Issues:
- Basically same ones as in the jekyll issue.
- How to handle
minutes/. It's gigantic and inappropriate to mix into a static site directly.- It is being updated on the main site AND on gh pages at https://json-ld.github.io/minutes/, but the logs note that 1.8G is beyond the 1G max and deploy might fail and it can take 20min+ to deploy. Another solution would be better. Also the "pre-wg" link is easy to not notice.
- How to handle the playground and playground php proxies.
- How best to host and handle updates.
- May need content type and cors fixes depending on hosting platform.
Notes:
- One approach for many issues is a subdomain using gh pages or something else for minutes, cloudflare for hosting, cloudflare workers for the proxy. I think Benjamin likes this idea.
- We also need to decide how to improve the playground. No one at all wants to work on it as is. Integrating a web app build step into the static site process is doable but perhaps not the best approach. I'm pondering the site code here being more content based and (ab)using a CDN or gh pages to host built playground app bundles and resources. Suggestions welcome on that.
Testing:
npm install
npm run serve
# open http://localhost:8080/ (or whatever URL it gives you)
Deploying json-ld-org with 
Cloudflare Pages
| Latest commit: |
a66b568
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://99ef1971.json-ld-org.pages.dev |
![cloudflare-workers-and-pages[bot] avatar](https://avatars.githubusercontent.com/in/85455?v=4 "Published by a pub.dev verified publisher"){kind=small}
Another issue with this conversion is how to handle all the .htaccess rules. If the target host is Apache then of course these will still work. But our likely new target is Cloudflare Pages, which does things differently.
$ git ls-files | grep .htaccess
.htaccess
contexts/.htaccess
images/.htaccess
spec/ED/.htaccess
spec/latest/.htaccess
test-suite/.htaccess
test-suite/tests/.htaccess
utils/.htaccess
- CF has
_redirectsand_headersfiles, which are simpler and limited. Some of the.htaccessrules can be handled with those files. (Untested, but looks promising) - The content type detection is trickier and I believe would require another
_functionsworker for some special cases. Effectively writing some js to do what apache rules do.
And while pondering these things, I'm wondering where this is headed.
- A good dev env is now rather complex.
- 11ty has the nice auto update dev server. But it's unaware of these htaccess or cloudflare rules. Making it difficult to actually test the entire site. To use the playground proxy (see other PR), you need to have a 11th proxy to a cloudflare wrangler dev server.
- Cloudflare wrangler could also serve the static dev site with its redirects and headers and functions. But then the auto update feature doesn't work. There may be some extra proxying to make that work. But then that would require two dev servers to do dev work, which seems a bit awkward.
- Should we make apache or cloudflare a build option? I suspect it will just flip which files get copied to the output. But that's two things to maintain and keep in sync.
@davidlehn per your concerns above, I think we can keep using 11ty separate from the hosting platform. Having 11ty in place makes working on (and redesigning) the site much simpler--where it's hosted can be a separate matter because 11ty generates static sites.
Cloudflare hosting can easily get more complex/tangled (just like the build tool nightmare that is modern JavaScript vs. ye olde <script> tag). It really depends on what we want to manage, pay for, support, etc. Most/all of the Cloudflare stuff, though, can also be defined descriptively (or at least via JS in the same repo) to avoid "blind"/hidden configurations available only via the Cloudflare Dashboard.
All that to say, how this is hosted is orthogonal to using 11ty.
@davidlehn how close do you think this is to shippable? I know #840 is part of the consideration here too. Just curious how we can get this over the finish line.
Since the WG is getting rechartered, it'd be super to get this in a space we can continue to improve, redesign, etc.
Happy to help finish it off...just not sure where we're stuck.
- 11ty has the nice auto update dev server. But it's unaware of these htaccess or cloudflare rules. Making it difficult to actually test the entire site. To use the playground proxy (see other PR), you need to have a 11th proxy to a cloudflare wrangler dev server.
- Cloudflare wrangler could also serve the static dev site with its redirects and headers and functions. But then the auto update feature doesn't work. There may be some extra proxying to make that work. But then that would require two dev servers to do dev work, which seems a bit awkward.
To untangle this, I've added a npm run pages option which can be used alongside npm run watch. That should provide the handy auto-building from 11ty while serving a localhost Cloudflare-like environment.
- Should we make apache or cloudflare a build option? I suspect it will just flip which files get copied to the output. But that's two things to maintain and keep in sync.
I don't think we should do this as it'd be a good deal of overhead to maintain/juggle both. Consequently, I've removed the .htaccess files in #855.
Once #855 is merged into this PR, I think this one can be marked "ready for review", and we can get a preview URL on Cloudflare sporting all the things for further testing from everyone here.
@davidlehn I think I've addressed most if not all of your concerns in https://github.com/json-ld/json-ld.org/pull/836#issuecomment-2103040505 via #855, so I've marked this as ready for review.
Please let me know what else it needs. Thanks!
Functional preview available at https://99ef1971.json-ld-org.pages.dev/
Clicking back through the site, I realized we never answered this one:
How to handle
minutes/. It's gigantic and inappropriate to mix into a static site directly.
- It is being updated on the main site AND on gh pages at https://json-ld.github.io/minutes/, but the logs note that 1.8G is beyond the 1G max and deploy might fail and it can take 20min+ to deploy. Another solution would be better. Also the "pre-wg" link is easy to not notice.
https://99ef1971.json-ld-org.pages.dev/minutes/ (in the preview) is current 404.
The simplest solution would be to setup a redirect from that URL to https://json-ld.github.io/minutes/ (where it's actually hosted).
Alternatively, if we think the json-ld.org URLs are important, we could proxy those URLs through a Cloudflare Pages Function (though that likely comes with some bandwidth costs)--which should not incur additional costs (afaict from https://developers.cloudflare.com/workers/platform/pricing/).
Or, similarly, we could push the files to a Cloudflare R2 public bucket (which supports large static file hosting similar to S3). However, that may eventually incur some cost or at minimum we'd have to be strategic in how we setup publishing the files to R2--so we don't ship all 1.8 GB every time a new change is made to the minutes...
My current vote would be a simple redirect setup to keep thing uncomplicated.
Thoughts @davidlehn @gkellogg?