cht-core
cht-core copied to clipboard
medic
patch: descendant of current contact
Description
Similar to ecsalomon's github issue rationale, mentioned below, we've been seeing an increased amount of CHWs creating duplicate contacts. This is a multi faceted problem, which has spawned quite a bit of conversation, as can be seen by visiting the forum thread linked below.
Backport of a CHT feature originally developed by latin-panda. Related to this forum thread attempting to combat duplicate data capturing. Which in turn is related to the original duplicate issue raised by ecsalomon.
License
The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.
@ChinHairSaintClair It's usually problematic to backport features, because when you get around to releasing the only way to do it is to do it is to use a patch in semver which is defined as "PATCH version when you make backward compatible bug fixes". This is done on purpose so people upgrading from 4.2.0 to 4.2.10 know that all changes are backwards compatible.
The ideal solution is you upgrade to the latest to get this and all other fixes and improvements. Is there anything blocking you from upgrading to the latest?
Hi @garethbowen, thank you for the info. There's a couple of concerns with regards to the upgrade. We're not just concerned with the stability of our app. but also with the effects downstream.
The most immediate concern is the necessity of deleted documents tombstones has been removed.
Downstream we use this to trigger db deletes.
Will we still be able to discern if a record has been deleted? And what happens to existing tombstones?
Then with the larger 4.4.x upgrade our dedicated DevOps, @sircollins, has raised the following concerns (many of these items' impact have yet to be investigated):
- Grunt dropped in favor of npm scripts - we've invested quite a bit of time into grunt tasks
- Couch DB upgrade impact on Data Engineering Team
- Downstream replication rewrite (persistent caching of purged doc ids)
- Update configuration deployment to use latest
cht-conf - Migrate from SW-Precache to Workbox
- Update nginx to latest
For a little background we use a couchDB connector that was written by our Data Engineer @juanpaulH. It uses the changes API to pull records into their landing, after which the data undergoes various manipulations via the medallion architecture. Finally, when the data has been sufficiently sanitized, it gets processed by stored procedures and pushed to a SQL database which gets used for reporting.
The abovementioned might not be an exhaustive list, as one can see, we're still very much in the investigation phase of the larger 4.4.x upgrade. Per our road map we're hoping to overcome this hurdle by June 2024, after which it should be much quicker to stay in step with CHT as new versions get released.
That being said, in the interim we're trying our best to find mechanisms that combat the duplication of data capture.
@ChinHairSaintClair If you're running grunt tasks to build the CHT then it sounds like you're already on a complete fork of the CHT. If that's the case then you should be free to apply this PR to your fork, and it's not necessary to merge it into the 4.2.x branch of the medic fork, right?
You've clearly done a lot of work separately which is making it difficult to keep up with the medic fork. I would need to know a lot more about your situation to understand how you're using things like tombstones to recommend a way forward. Are these changes that should be contributed upstream for the benefit of the whole community? Are these things that could be supported by supported endpoints and plugins so you don't need a fork?
Hi @garethbowen As you've noted, it's not necessary to merge it into the 4.2.x branch of Medic. However, avoiding this merge increases the number of files in our 'patch' folder, and subsequently, complicates the upgrading process between CHT versions. Now that you mention it, backporting this feature is more of a 'convenience improvement' than a necessity.
That's very fair. Perhaps in the near future we can arrange a call for a quick run through of our setup, with the resources responsible for those pieces of work.
With regards to the tombstones:
We've been running tests on our local and non-prod environments to test the 4.3.x upgrade, which pertains to the tombstone change mentioned above. As mentioned in this CHT forum thread, we seem to have a way forward that aligns with our requirements. We'll know more within the next week or so.
@ChinHairSaintClair Sounds great. Feel free to email me at [email protected] and we can set up a call!