blog: prerendering and Rendertron include default meta tags instead of those for the post

[Splaktar]

Expected outcome

When sharing blog posts, the G+, Twitter, and Facebook meta tags like og:url, og:type, og:title, og:image, og:description, fb:app_id should be defined and allow for a rich sharing experience.

Actual outcome

Facebook:

G+:

Twitter:

The data has a title and image field, so at minimum it seems like those meta should be set properly on blog pages

"blog": {
  "gold-sponsor-raywenderlich": {
      "title": "Gold Sponsor: raywenderlich.com",
      "published": "2019-01-15",
      "backgroundColor": "#3647a5",
      "source": "https://devfestflorida.org",
      "image": "https://devfestflorida.org/images/posts/raywenderlich-team.jpg",
      "brief": "...",
      "content": "..."
    },
...

Steps to reproduce

1. Create a new blog post entry with all of the appropriate fields
2. Deploy site w/ any required images
3. Upload blog post to Firestore
4. Copy Blog post production URL
5. Use Blog post URL on Twitter/Facebook/G+

[Splaktar]

Based on https://github.com/gdg-x/hoverboard/blob/a62532f97a6193869da5d06708ea41ad5a8b4462/src/pages/post-page.html#L77-L84

It seems like this should work. I'm not sure why it is not functioning for us.

[Splaktar]

Is this solved by https://github.com/gdg-x/hoverboard/blob/master/docs/tutorials/seo.md?

Also since the change to run on App Engine Standard instead of Flex it seems like this should cost a lot less than the $60 a month mentioned in the tutorial. Does anyone have experience with the cost on GAE Standard?

[Splaktar]

For firebase functions:config:set site.domain="YOUR_SUB_DOMAIN_HERE.firebaseapp.com" should this always be using firebaseapp.com even if you have a custom domain for your site like devfestflorida.org?

[Splaktar]

OK, I've got Rendertron deployed to GAE Standard and it seems to be working now. However, I'm still not seeing the meta tags get updated. They are staying set with the default site meta tags and not the blog post title or image.

Facebook doesn't 404 anymore:

Twitter renders the default instead of a blank:

G+ is the same, default meta is rendered but not the blog post meta.

[ozasadnyy]

@Splaktar did you setup rendertron?

[Splaktar]

@ozasadnyy yes, as mentioned in https://github.com/gdg-x/hoverboard/issues/543#issuecomment-454997764.

[Splaktar]

In the browser looks good, but not via social media:

[Splaktar]

I see it working on your blog pages. One difference that I see is this:

Input URL	arrow-right	https://devfestflorida.org/blog/posts/gold-sponsor-raywenderlich/
og:url Meta Tag	arrow-right	https://devfestflorida.org/

Your blog posts don't show that redirect in the Facebook debugger.

[ozasadnyy]

For firebase functions:config:set site.domain="YOUR_SUB_DOMAIN_HERE.firebaseapp.com" should this always be using firebaseapp.com even if you have a custom domain for your site like devfestflorida.org?

@Splaktar yes. Have you added that and rerun functions?

[Splaktar]

@ozasadnyy yeah, it's set to devfestflorida.firebaseapp.com and functions have been redeployed via firebase deploy --only functions. I just redeployed them again to be sure, still the same issue.

[Splaktar]

Oh btw, we're on 2.0.0. I'm working on trying to rebase to 2.1.0 to see if it helps.

[Splaktar]

Updated everything to 2.1.0 from latest master, but unfortunately it hasn't helped with this issue.

[ozasadnyy]

Which rendertron version are you using? If v2, please try to downgrade to v1 https://github.com/GoogleChrome/rendertron/releases/tag/v1.1.1

[Splaktar]

@ozasadnyy we're running 2.0.1 of Rendertron. I'll give that a shot when I get a chance, thank you.

[Splaktar]

When I hit Rendertron via https://devfestflorida.appspot.com/render/https://devfestflorida.org/blog/posts/gold-sponsor-raywenderlich/, I get the following error that looks like it may be related:

Access to resource at 'https://devfestflorida.org/manifest.json' from origin 'https://devfestflorida.appspot.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

I am going to see if configuring CORS on Firebase hosting will help.

[Splaktar]

Adding the following to my Firebase Hosting headers fixed the CORS error on Rendertron, but it still doesn't render with the correct meta tags:

          {
            "key": "Access-Control-Allow-Origin",
            "value": "https://devfestflorida.appspot.com"
          }

[Splaktar]

On Rendertron v1.1.1 I always get this error:

10 second time budget limit reached.
          Attempted rendering: https://devfestflorida.org/blog/posts/gold-sponsor-raywenderlich/
          Page load event fired: true
          Outstanding network requests: 1

I bumped the limit to 20s and that didn't help. Our page takes about 1.5s to load so this just seems to be failing somewhere in Rendertron w/o reporting an error.

This might be caused by it still waiting for a Firebase Auth or Firestore interval that never completes.

[Splaktar]

Here's the Rendertron that we're using: https://github.com/DevFestFlorida/rendertron. I made some updates to dependencies and tweaked the app.yaml to save some money by scaling to zero. With or without the updates, it still has the same issue with the og:url and other fields staying as the defaults.

Our event is Saturday, so I don't plan to keep spending more hours trying to fix this (over 10 hours so far).

[jcf120]

I don't know if the current deployment of gdg-x/hoverboard is up to date, but it also fails to render correctly on this public rendertron instance:

https://render-tron.appspot.com/render/https://hoverboard-master.firebaseapp.com/blog/posts/dfua17-announced/

I've noticed that rendertron only waits until puppeteer signals networkidle0, which I suspect comes too early hoverboard's page load.

[ozasadnyy]

@jcf120 I shut down Rendertron service for now as it's quite expensive to maintain. Looking for another solution. Do you have something on your mind?

[jcf120]

Think I've finally got a lead. When running rendertron with headless mode disabled, I get the following stacktrace in the inspector:

webcomponents-sd-ce.js:169 Uncaught TypeError: b.indexOf is not a function
    at hf (webcomponents-sd-ce.js:169)
    at Pf (webcomponents-sd-ce.js:189)
    at Y.n.styleElement (webcomponents-sd-ce.js:185)
    at Y.n.styleSubtree (webcomponents-sd-ce.js:190)
    at Object.styleSubtree (webcomponents-sd-ce.js:195)
    at HTMLElement.updateStyles (hoverboard-app.html:6366)
    at HTMLElement._setToolbarSettings (hoverboard-app.html:28270)
    at Object.runObserverEffect [as fn] (hoverboard-app.html:2812)
    at runEffectsForProperty (hoverboard-app.html:2757)
    at runEffects (hoverboard-app.html:2723)

[jcf120]

Think I've got it. The polyfill can't handle non-string css props. Here's the culprit:

'--hero-logo-opacity': settings.hideLogo ? 0 : 1,

I'll open a PR

[jcf120]

(PR opened)

[jcf120]

@jcf120 I shut down Rendertron service for now as it's quite expensive to maintain. Looking for another solution. Do you have something on your mind?

Sorry, missed your comment before. I was trying to fix for the sake our own fork. We're only 3½ weeks from our conference. How far off do you think the next solution is?

[pif]

What if instead of rendering and serving content from rendertron, we serve prerendered static website to robots?

E.g.:

1. Prerender website into static locally or on CI (e.g. with rendertron or prerender.
2. Deploy rendered static content folder somewhere.
3. Deploy simple middleware as a Cloud Function or Cloudflare Worker, which would serve static content for robots and PWA to other clients.

[jcf120]

At what stage would this prerender of static content run? Much of the data comes from collections in firestore, so presumably most db mutations would need to trigger a render and deployment of the static content.

Could it possibly be simpler to write a firebase function for serving meta tags from the db more directly? (I'm assuming that we're only interested in serving meta data to bots, which I may be totally wrong about.) Aside, I'm considering something similar for adding an rss feed to the RadicalxChange hoverboard fork.

[ozasadnyy]

@jcf120 we need metadata for crawlers and social media bots to get a meaningful preview. We totally can write a firebase function which based on data from DB will generate metadata and return it for bots.