influxdb icon indicating copy to clipboard operation
influxdb copied to clipboard

In 2.x, seems series ids may be possible to be incomplete in series id set cache?

Open Rachelint opened this issue 8 months ago • 0 comments

Steps to reproduce: I am reading 2.x source codes recently, and found series ids seems can be incomplete in some cases? I am not so sure about it...

The tagValueCache's normal filling path seems like:

  • Search series ids by tag key + tag value from each partition(include current inactive log file, active log file, index file) in Index::TagValueSeriesIDIterator.
  • Build the multi level index measurement -> tag key -> tag value -> series ids in Index::TagValueSeriesIDIterator.
  • Append the new log(include new series and its index entries) into active log file in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Insert the new series id if the measurement -> tag key -> tag value -> series ids path above exist in cache in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.

However, due to multithreading, the order of steps can be:

  • Search series ids by tag key + tag value from each partition(include current inactive log file, active log file, index file) in Index::TagValueSeriesIDIterator.
  • Append the new log(include new series and its index entries) into active log file in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Try to Insert the new series id, but found the measurement -> tag key -> tag value -> series ids path non-exist in cache in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Build the multi level index measurement -> tag key -> tag value -> series ids in Index::TagValueSeriesIDIterator.

And in the later query, because the measurement -> tag key -> tag value -> series ids exists, it won't try to search from files again, so the series ids have no changes to become complete...

Expected behaviour: Complete series ids in cache.

Actual behaviour: Maybe incomplete series ids in cache.

Environment info:

Config:

Logs:

Rachelint avatar Jun 12 '24 17:06 Rachelint