Stale JSON data loaded on initial page load

[paulschreiber]

We are seeing stale JSON data loaded on initial page load.

Example (9/27/2018):

• Live blog: https://fivethirtyeight.com/live-blog/brett-kavanaugh-hearing/
• at 10:45am EDT, an initial request is made for https://fivethirtyeight.com/wp-json/liveblog/v1/191397/get-entries/1/2446-1538058250

This loads items through 10:24am entry. After a delay, a request is made to https://fivethirtyeight.com/wp-json/liveblog/v1/191397/entries/1538058251/1538059580/ which loads entries through 10:41am.

[philipjohn]

Hi Paul, can you provide more details about the JSON itself? What do you see and where does that become stale?

[paulschreiber]

What do you see

You can click the links above to see what we saw.

where does that become stale?

I'm not sure what you are asking.

[paulschreiber]

Could be related to #489.

[paulschreiber]

• 10:45am is 1538059500
• first GET request is for 1538058250, which is 10:24am
• second GET request is for 1538059580, which is 10:46am

The initial request comes from:

export function getEntries(page, config, newestEntry) {
  const settings = {
    url: `${config.endpoint_url}get-entries/${page}/${newestEntry.id || config.latest_entry_id}-${newestEntry.timestamp || config.latest_entry_timestamp}`,
    method: 'GET',
  };

  return secureAjax(settings);
}

Subsequent requests come from:

export function polling(newestEntryTimestamp, config) {
  let timestamp = getCurrentTimestamp();
  // Round out the timestamp to get a higher cache hitrate.
  // Rather than a random scatter of timestamps,
  // this allows multiple clients to make a request with the same timestamp.
  const refreshInterval = parseInt(config.refresh_interval, 10);
  timestamp = Math.floor(timestamp / refreshInterval) * refreshInterval;

  const settings = {
    url: `${config.endpoint_url}entries/${(newestEntryTimestamp + 1) || 0}/${timestamp}/`,
    method: 'GET',
  };

  return secureAjax(settings);
}

Can someone look over the timestamp calculations and see if there's an obvious error?

[christianc1]

Is it possible that a page cache is causing window.liveblog_settings.latest_entry_id and window.liveblog_settings.latest_entry_timestamp to contain stale data?

[paulschreiber]

Would the page cache be limited to 5 minutes? This is a 21-minute gap.

[philipjohn]

I wonder if this has something to do with the time diff code we removed recently... In any case, we're likely going to be changing the requests to be much more normalised, including removing the timestamps.