Redirection lambda function regex missing .map, .json and whatsapp

[YuriScarbaci]

The lambda testing wheter to set the header for prerender.io or not isn't currently testing:

1. user-agent for whatsapp crawler ( should be something like WhatsApp/2.19.81 A )
2. .json files
3. .map files

[YuriScarbaci]

this can be fixed by modifying the lambda function for SetPrerenderHeader

this is the part that needs to be changed:

    if (user_agent && host) {
      var prerender = /googlebot|adsbot\-google|Feedfetcher\-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value);
      prerender = prerender || /_escaped_fragment_/.test(request.querystring);
      prerender = prerender && ! /\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri);
      if (prerender) {
        headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: '${PrerenderToken}'}];
        headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}];
        headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}];
      }
    }

so basically adding |WhatsApp in the first regex

and

|map|json to the third regex

[jinty]

Hi Yuri,

Thanks for the info. The change looks good. Would you be able to make a pull request with it?

Regards, Brian

On Thu, 31 Oct 2019 at 18:49, Yuri Scarbaci [email protected] wrote:

this can be fixed by modifying the lambda function for SetPrerenderHeader

this is the part that needs to be changed:

if (user_agent && host) {
  var prerender = /googlebot|adsbot\-google|Feedfetcher\-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value);
  prerender = prerender || /_escaped_fragment_/.test(request.querystring);
  prerender = prerender && ! /\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri);
  if (prerender) {
    headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: 'ZOPzswpcTJUhqRXNX2FC'}];
    headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}];
    headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}];
  }
}

so basically adding |WhatsApp in the first regex

and

|map|json to the third regex

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jinty/prerender-cloudfront/issues/14?email_source=notifications&email_token=AAEHDGNWDLCPMLUFA24MCCDQRMEAHA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECYPDYQ#issuecomment-548467170, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEHDGIHC5EPJGZFJWAUP63QRMEAHANCNFSM4JHNAJMQ .

[YuriScarbaci]

I'm not sure enough about the comment in the lambda function saying that the version is to be changed below.

Is it enough for me to change what I stated or should I change something more in the cloud formation yaml?

On Thu, Oct 31, 2019, 19:48 Brian Sutherland [email protected] wrote:

Hi Yuri,

Thanks for the info. The change looks good. Would you be able to make a pull request with it?

Regards, Brian

On Thu, 31 Oct 2019 at 18:49, Yuri Scarbaci [email protected] wrote:

this can be fixed by modifying the lambda function for SetPrerenderHeader

this is the part that needs to be changed:

if (user_agent && host) { var prerender = /googlebot|adsbot-google|Feedfetcher-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value); prerender = prerender || /escaped_fragment/.test(request.querystring); prerender = prerender && ! /.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri); if (prerender) { headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: 'ZOPzswpcTJUhqRXNX2FC'}]; headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}]; headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}]; } }

so basically adding |WhatsApp in the first regex

and

|map|json to the third regex

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/jinty/prerender-cloudfront/issues/14?email_source=notifications&email_token=AAEHDGNWDLCPMLUFA24MCCDQRMEAHA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECYPDYQ#issuecomment-548467170 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAEHDGIHC5EPJGZFJWAUP63QRMEAHANCNFSM4JHNAJMQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jinty/prerender-cloudfront/issues/14?email_source=notifications&email_token=ACQWZVWXBLXVXKH55WV6LODQRMR6HA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECY3ONI#issuecomment-548517685, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQWZVRG3CEHLICE4QEGY6TQRMR6HANCNFSM4JHNAJMQ .