How to remove/merge duplicates in the database (athletes/clubs)

[espinr]

During the monthly call on Feb 15, Andy R raised an open question about the best way for cleaning the database, and avoiding duplicates when inserting new entries. In concrete with competitors, athletes and clubs.

So, please list best practices and ways to detect exact and near duplicates.

For instance: Reversing dates (month, day), name/surname, etc.