public contract pages have different expiry

[guiguilechat]

Bug

When fetching several pages on the public contract endpoint, sometimes those pages have a different expiry date

Request

https://esi.evetech.net/v1/contracts/public/10000002/?&page=1 and https://esi.evetech.net/v1/contracts/public/10000002/?&page=3

My program threw a warning (repeatedly, like for 2 minutes until I close it)

WARN  f.g.jcelechat.jcesi.ConnectedImpl - mismatching page cache data [url=https://esi.evetech.net/v1/contracts/public/10000002/?&page=3 Expires=[Sat, 28 Sep 2019 00:45:32 GMT]] 
with first page [url=https://esi.evetech.net/v1/contracts/public/10000002/?&page=1 Expires=[Sat, 28 Sep 2019 00:45:33 GMT]]

Explanations

Pages

Pagination refers to having the dataset of a path split among several pages. The client requires the first page, and in the header or the response he gets the number of page, as well as the cache expiry information (among other). Other pagination methods exist but they are not the issue.

Synchronizing the pages

Possible issue

The pages for a path are cached by the server, and the server cache can be altered between the access to the first page and access to the second page. This alteration can produce a representation in the client's memory that does not match the server data at any time.

For example, let's consider the following case :

• server-side page 1 contains the value AB, page 2 contains the values CD
• client requests page 1, gets AB and knows he needs to fetch page 2
• server gets a cache update, which makes page 1 BC and page 2 DA
• client requests page 2 and received DA. He therefore considers the aggregated resource is ABDA while the server's aggregated resource is DCDA.

Though not very likely, this can happen and produce on the client a random value for any paginated resource.

Avoid with Expires

To avoid this case, the client needs to be sure the pages he receives all belong to the same aggregate data.

As you can see in the request examples I provided, the only header data that should change when the aggregated data changes, and remain the same between the pages of that same aggregate, is the expires one.

When the Expires can be used to check for synchronization, unmatching Expires should result in the client considering the request as being a 503 service-unavailable with a retry-after of 1

If the Expires changes between pages, then this desync issue can't be avoided anymore.

Fix

Either be sure all the pages have the same exact expiry (by setting the expiry of the pages 2+ to the expiry of the first one). Or add a specific header for this (eg x-page-aggregate). I have no idea which one is best. If you have better solution, please explain.

Checklist

Check all boxes that apply to this issue:

• [ ] Bug description is provided
• [ ] Request path is provided
• [ ] Response status code is provided
• [ ] Response headers are provided
• [ ] Response body is provided
• [ ] Expected response is provided

[guiguilechat]

should be fixed by CCP_blakara as of today. To close in a few week if no more issue.