elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

Do not produce infinity values in synthetic source for range fields

Open lkts opened this issue 9 months ago • 4 comments

Currently if one of range bounds of float_range or double_range field has a default value, it is represented as "-Infinity" or "Infinity". This is bad since infinities are not valid input values for these range fields and therefore reindexing such document will fail. One of the main goals of synthetic source is to be able to reindex documents so this needed to change.

This PR changes the implementation to always explicitly write null to synthetic source if range bound has a default value (see rest tests). Note that for consistency this is applied to all range types and not only to float_range and double_range.

As a nice bonus it allows us to not put "weird" values in source like "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" and "-292275055-05-16T16:47:04.192Z".

One downside here is that if customer explicitly provides a value that is equal to default range bound, synthetic source will still contain null. I don't think this is a common use case and there is nothing we can really do (except storing some additional metadata).

Contributes to #106460.

lkts avatar May 16 '24 00:05 lkts

Pinging @elastic/es-storage-engine (Team:StorageEngine)

elasticsearchmachine avatar May 16 '24 00:05 elasticsearchmachine

Should this be treated as a breaking change? I believe it's desirable since we want docs that can be reindexed, but it should go through the committee nonetheless?

kkrik-es avatar May 16 '24 08:05 kkrik-es

Synthetic source for ranges is not released yet so it shouldn't be breaking #107081.

lkts avatar May 16 '24 15:05 lkts

@elasticmachine update branch

lkts avatar May 16 '24 17:05 lkts