Prevent and/or merge duplicate contacts

[ecsalomon]

Is your feature request related to a problem? Please describe. This excellent report by @marialma shows that duplicate contacts are an issue for a non-trivial amount of CHWs in some deployments. Maria has enumerated a large number of issues that arise for care delivery, privacy, and analytics from this. For example:

• For client-initiated health assessment, duplicate contacts create issues of linking phone numbers to patients or households
• Recreating a patient who moves to a new household means that the CHT and analysts lose the history of that patient, which may impact care delivery (e.g., lose the context of a registered high risk pregnancy)

Describe the solution you'd like There are several potential solutions:

1. When entering a new contact, search existing contacts for exact or extremely close (say, Levenshtein distance = 2) names and ask the CHW to confirm that they want to create a new contact.
2. Allow CHWs to "move" contacts between parent contacts (e.g., when someone moves to a new household within a catchment area)
3. For older deployments, let's say a data scientist sets up an entity matching service on the backend to identify likely duplicates, allow a CHW/supervisor to review and confirm duplicates and merge them (so that all documents become associated with the merged/new contact) and deconflict contact info (e.g., date of birth).

Describe alternatives you've considered There are likely some training solutions (e.g., advising CHWs of the problems created by duplicate contacts). But training will never address everything.

Additional context

[ecsalomon]

See also #6309

[jkuester]

Additional conversation about the need for this feature on this forum thread!

[ChinHairSaintClair]

We've opted to use the descendant-of-current-contact feature, although it doesn't use Levenshtein distance, hoping it will help mitigate duplicate captures.

We've also been looking into using the cht-conf tool's contact-move functionality to consolidate records where possible - as a method of "damage control."

After reviewing the death reporting functionality provided by CHT, we wondered if it is possible to create a custom transition appending some flagging property to a record if its immediate siblings are similarly named. Then, a task would examine this property and flag entries for correction. As noted by @jkuester, this will require some core changes and the CHWs' devices to be online and successfully synced. Additionally, initiating a contact edit form directly via the task will not load the required edit info. This will necessitate an ephemeral form that redirects to the edit form.

Originally mentioned on this CHT forum thread.

[kennsippell]

Hey @ChinHairSaintClair regarding the use of move-contacts in cht-conf to merge duplicates, check out https://github.com/medic/cht-conf/issues/373. The issue links to a hacky branch with a version of cht-conf that merges duplicates which might make a good starting point. This ability to merge duplicate contacts is badly needed functionality, so would welcome collaboration there.