rendertron
rendertron copied to clipboard
Rendertron not serving files as https
I have a client-side SPA PWA (uses https) and trying to use Rendertron to serve files to crawlers. When navigating to my website I receive an error as manifest.webmanifest is requested with http protocol and the page stops loading. manifest.webmanifest | (blocked:mixed-content)
My index.html is correctly requested over https but then for some reason the next file (manifest) is requested over http.
(it does not act that way when not going through rendertron)
As you can see the user agent here is set to Googlebot. When I remove that, rendertron.makeMiddleware apparently calls next in my express app and the static files are served correctly over https, the website loads completely.
The rendertron instance is an out of the box version with no changes made. I cloned the git project and uploaded to google cloud and the UI works correctly. https://rendertron-dynamic-render.ew.r.appspot.com/render/https://MYDOMAIN.com/%3Fdom%3Dshady%26wc-inject-shadydom%3Dtrue The above is the instance and works correclty. When adding the url through the ui I use the Shady Dom params but in the express app I add "injectShadyDom: true" instead.
app.use(rendertron.makeMiddleware({ proxyUrl: 'https://rendertron-dynamic-render.ew.r.appspot.com/render', userAgentPattern: BOT_UA_PATTERN, injectShadyDom: true }));
Is there a configuration I need to change in the out of the box Rendertron version?
One thing I less understand but I will mention is regarding https: My domain is set up to use https and works correctly even when serving static files through my express app (not as bot). That is with me using: const app = express(); app.listen(PORT...
For some reason this works if I set port to 80 or if I set it to 443. This works identically if I use: http.createServer(app).listen(PORT... instead.
Is my problem that I have to be using also: https.createServer(options, app).listen(443
or something I am missing that I have to configure in the Rendertron app? Thank you
*** Update added - protocol: 'https' to configuration (still receive same results as above) app.use(rendertron.makeMiddleware({ ... protocol: 'https' }));
*** Additional Update On second look, it seems like only the manifest.webmanifest is being served over http while all other files are served over https. Is there something regarding a PWA manifest file that needs handling or configuring in Rendertron?
Also can you make it clear if rendertron.makeMiddleware sends the file and does not call next? The reason I am asking is I see a log after the rendertron-middleware is used and just before my express app serves a static file. (But I do get the rendertron page correctly with no JavaScript on the page)
Do you happen to have a sample URL where we could see this behaviour and try to debug it?
As of now, I don't know where in the stack the issue stems from, because Rendertron doesn't have any special https settings nor does it treat specific files differently. All the middleware does is route any request to your express.js app that comes in with a bot user-agent to the Rendertron server you set up and then takes the response from that and pipes it back into your express app's response.