elasticsearch
elasticsearch copied to clipboard
elastic
Lookup runtime sub-fields to throw error when queried or aggregated on
The lookup runtime field enriches the top hits with fields retrieved from a secondary index. This is purely a fetch phase enrichment at the moment, and users may get surprised that the sub-fields that are fetched from the secondary index are not searchable nor aggregatable. Effectively those are treated like unmapped fields. Would it make sense to throw an error when they get queried or aggregated on?
Posting thoughts from a discussion with @dnhatn : the lookup field does throw error when you try to aggregate/query it. What users may stumble upon is trying to query/aggregate on the sub-fields that are fetched from the secondary index. Those are not in the mappings and will be treated like unmapped fields. Also, these are not returned by field_caps, which is good for consumers that do look at field_caps, but some users could be lead to believe that the lookup runtime field performs a join hence they try to query the field that they are joining on.
The fields fetched from the secondary index could be expressed with a wildcard expression, hence the only way we could throw an error when these are queried/aggregated on would be to make the lookup runtime field own its entire namespace (a la flattened field) and have a dynamic mapped field type for its sub-fields whose only purpose is to throw errors. These would shadow any field with same name already present in the mappings, and it seems overkill if the only purpose is to throw an error. We will look at improving the docs and making it even more explicit that these sub-fields can't be queried nor aggregated on.
