expirationd icon indicating copy to clipboard operation
expirationd copied to clipboard

Investigate how to support the use-cases from indexpiration

Open akudiyar opened this issue 4 years ago • 2 comments

Currently, expirationd does not support iteration over a (secondary) index (since the expriration check callback must be applied to each tuple from a space). This creates a situation where a full scan is applied instead of selecting tuples from some start index value.

TODO:

  • [x] Investigate how indices are used in the current expirationd implementation (check how the primary index is used, is it possible to iterate over a complex primary index, simple or complex secondary index)
  • [x] Propose a RFC for changing the API to support iterating over a specified index or a field (the minimal satisfying index must be chosen in this case if it exists)
  • [ ] Check how the module is initialized: is it possible to reload the configuration and any created fibers or the internal state at runtime, thus supporting the "hot reload" feature
  • [ ] Propose a RFC for changes to support the "hot reload" feature

akudiyar avatar Apr 12 '21 18:04 akudiyar

The result of analyzing the existing code:

  1. Now there is a hardcode implementation of iterations by index zero, i.e. by primary. Secondary or complex indexes cannot be used at this time. https://github.com/tarantool/expirationd/blob/29d1a25962bb5633d094bf7b016480690226da14/expirationd.lua#L91-L100
  2. hot reload of the configuration is possible, since we have direct access to the variables that are used in the fibers. https://github.com/tarantool/expirationd/blob/29d1a25962bb5633d094bf7b016480690226da14/expirationd.lua#L30 and so we already have a function to reload the expirationd module - update (). https://github.com/tarantool/expirationd/blob/29d1a25962bb5633d094bf7b016480690226da14/expirationd.lua#L515-L519
  • [x] need to write a test for hot reload

ArtDu avatar Apr 13 '21 06:04 ArtDu

We don't need this, since there is indexpiration already. These modules have strongly different approaches: by index and by callback. Mixing them up in one module will lead to many misuses.

I see no reason to implement it in expirationd. Different modules cause the user to choose between patterns: either full-scan with the arbitrary condition inside user function or very precise index scan by index with a timestamp.

Mons avatar Apr 13 '21 16:04 Mons