elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

Published 20 hours ago • verified publisher icon elastic

CCR requests complete cluster metadata from remote cluster

Open DaveCTurner opened this issue 3 years ago • 1 comments

A couple of places in CCR request the metadata from the remote cluster, mostly in order to see if there are any change to the set of interesting indices:

https://github.com/elastic/elasticsearch/blob/12ad399c488f0cc60e19b5e1b29c6d569cb4351a/x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/AutoFollowCoordinator.java#L255-L268

https://github.com/elastic/elasticsearch/blob/12ad399c488f0cc60e19b5e1b29c6d569cb4351a/x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/TransportPutAutoFollowPatternAction.java#L118-L128

The metadata includes all index settings and mappings and can be quite expensive to send over the wire in a big cluster. I think we should either move this computation onto the leader cluster or else work out a way to be more parsimonious with what gets sent back to the follower.

Relates #77466

DaveCTurner avatar Oct 29 '21 17:10 DaveCTurner

Pinging @elastic/es-distributed (Team:Distributed)

elasticmachine avatar Oct 29 '21 17:10 elasticmachine