dfed
Enable performant checking of number of messages in cache
Goals
- Enable determining the number of messages in a cache without reading the entire cache every time.
Non-Goals
- Allow a CacheAdvance to enforce a limit to the number of messages in a cache.
- Disallow reading the entire cache to determine the number of messages.
Design
We should consider storing a numberOfMessages: UInt64 in the FileHeader. If we position the numberOfMessages field directly after the offsetInFileAtEndOfNewestMessage field, we will be able to update the values for offsetInFileAtEndOfNewestMessage and numberOfMessages atomically after we write a new message to disk, ensuring that this new value is always correct.
We'd also need to add a func numberOfMessages() throws -> UInt64 to CacheAdvance.
We could also store the numberOfMessages in memory in addition to in the file header. This would enable lightning-fast checking of the number of values in our cache.
Upgrading the file header to have this information will be tricky, since the number of messages can not be determined without reading all messages on the disk. We may need to store a sentinel value (proposal: UInt64.max, since we can never have that many messages in a single CacheAdvance), and if we encounter that value in the header when asking for numberOfMessages(), we'll need to calculate it by hand, store it, and then return it.
