No option for backing up from primary

[acheccuc]

We have a use case where backup needs to be taken on the data center hosting the replica set primaries, while the secondaries are geographically another continents. I read through the documents, but I didn't find any option to specify primary reads. Apparently this option is statically passed to mongodump:

mongodump_flags.extend(["--readPreference=secondary"])

in MongodumpThread.py, along with a secondary hostname.

Any plan to add flexibility on this aspect?

Thanks

[timvaillancourt]

Thanks @acheccuc,

Our goal was to make the tool as impact-proof as possible by avoiding the Primary at all costs, however I think it's reasonable to add an override option for the situation you're describing.

This will need some changes to more than one area of the code, however. I expect to get to this in version/milestone 1.3.0: https://github.com/Percona-Lab/mongodb_consistent_backup/milestone/3.

[Bastian82]

Would be nice to have a option to avoid cross dc traffic. Now, during scoring process there is possibility that backup server located in A location, pick up SECONDARY from location B. In our environment we have RS including nodes from A and B locations. Maybe add lowest ping into scoring algorithm?

[dbmurphy]

I think tags would be more effective here to have it only consider candidates in a location you want, however mongodump has no such logic so I would ask is it any worse than dump would be ?

Sent from my iPhone

On Dec 1, 2017, at 11:49, Krzysztof Grzempa [email protected] wrote:

Would be nice to have a option to avoid cross dc traffic. Now, during scoring process there is possibility that backup server located in A location, pick up SECONDARY from location B. In our environment we have RS including nodes from A and B locations. Maybe add lowest ping into scoring algorithm?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[timvaillancourt]

@dbmurphy recent mongodump DOES support read preference tags and mongodb_consistent_backup supports using tags since 1.2.0: https://github.com/Percona-Lab/mongodb_consistent_backup/pull/219.

The only missing piece for this issue is we don't allow a primary to be used, I hardcoded that read preferences (tags or not) can only be 'secondary'. All that's needed is adding a new flag like --use-primary that overrides that in these rare cases.

After this is added this deployment just needs to use the new flag to allow primaries combined with "--replication.read_pref_tags datacenter:something".

[Bastian82]

Right, having possibility of declaring tags (i assume mongo-consistent-backup use "--readPreference=secondary" as mentioned above) would do the job...

[timvaillancourt]

Correct, it's a pretty simple change that mostly involves calling this method with 'primary':

https://github.com/Percona-Lab/mongodb_consistent_backup/blob/master/mongodb_consistent_backup/Backup/Mongodump/MongodumpThread.py#L72-L76

And skipping most or all of the .find_secondary() work in Replication/Replset.py.

This functionality will require mongodump 3.2.0 or newer, FYI.