[New Feature] Tracking and Configuration of Candidate's Diagnosis Evolution

[jesscall]

Brief summary of changes

This PR adds a way for project administrators to define and configure diagnosis trajectories for their project. The aim of this new feature is multifaceted; it allows study coordinators to

1. Track the evolution of a patient's diagnosis over time
2. Determine (& query) the candidate's latest diagnosis is without having to manually compare the longitudinal data

Interest for this feature was expressed by multiple projects during the DCC meeting on July 7th.

---

What is included in this PR?

1. A configuration page for diagnosis evolution

2. A tab in candidate parameters to view details on the evolution of the diagnosis.

3. Logic to propagate the latest diagnosis data as instruments are updated. 4. A script that all projects should run when configuring Dx Trajectories and updating their instances. 5. Updates the demographic import to couchDB so the latest diagnosis can be queried.

---

Info

A Diagnosis Trajectory is defined by a source field, or group of source fields, that are found within the test battery. A patient's Diagnosis Evolution is defined by the progression of changing diagnoses over time. The evolution is determined by the order of configured Diagnosis Trajectories.

There are two ways that the Latest Diagnosis of the candidate can be updated. First, the script in this PR can be run. Secondly, when ANY instrument part of a configured diagnosis track is saved, the latest diagnosis is determined and updated in the candidate table.

EX:

• [x] Have you updated related documentation?

Testing instructions (if applicable)

1. fetch jesscall && git checkout jesscall/2021_07_27_diagnosis_evolution
2. cat SQL/New_patches/2021-07-28_diagnosis_evolution.sql | mysql
3. make dev
4. Follow instructions in configuration & caniddate parameters TestPlan.md files included in this PR
5. After setting up the configuration for a diagnosis evolution please do the following:
6. php tools/data_integrity/update_candidate_latest_diagnosis.php
7. Ensure that updating data in a configured instrument updates the latest diagnosis in the candidate table (if it's the latest) and the candidate parameters tab.

Link(s) to related PR(s)

• Blocked by #7514

[jesscall]

Hey @zaliqarosli I'm working on reactifying this PR as well as adding a proper endpoint as per @driusan request in a private convo.

I'll try to get to your feedbacks afterwards and see what is still relevant.

As for the rendering of diagnosis for QPN, that's a tough one. I imagine they are dropdown options? That sort of information isn't captured by data dictionary / parameter type. I guess you could hard-code the mapping or use grep on the instrument linst file... But these are both pretty hacky solutions. Maybe @ridz1208 has a suggestion

[jesscall]

@zaliqarosli I've updated the PR. It is now reactified, using a php endpoint rather than ajax and I've also addressed your comments :) Feel free to re-review at your convenience

@driusan your requested changes have been implemented

[jesscall]

should the Source Field be the database field name without the instrument prepended to it? i believe that was how it was implemented before reactification

@zaliqarosli

So in switching from templates to React, the datalist is implemented differently with our predefined react components. Before, you could search by the field name or the table name because both values were displayed in the select option. With the react component you can only have one of the two. So either:

1. The field name alone - EX: weight_lbs (parameter_type -> SourceField)
2. The compounded table name and field name - EX: bmi_weight_lbs (parameter_type -> Name)

I opted for (2) to make searching by instrument an option for the user. If you want this changed, let me know.

[jesscall]

Latest Changes:

• Added timestamp to track last update of latest diagnosis
• reverted back to using the sourceField names that are not prepended with instrument name.

[jesscall]

Refactored this PR to track the diagnosis evolution as a history rather than the current latest diagnosis. Also implemented @ridz1208 suggestion for showing the confirmation status of a diagnosis (whether the diagnosis comes from an approved visit or not).

---

[driusan]

@jesscall what is the status of this?

[jesscall]

@driusan ready for testing and review!

edit: rebased-

[jesscall]

@driusan @zaliqarosli @ridz1208 can someone help with phan ?

[zaliqarosli]

TO-DO:

• [x] implement Camille's comments
• [x] change Trajectory Name 409 Conflict to case-insensitive

[CamilleBeau]

@zaliqarosli Changes made!

[CamilleBeau]

@zaliqarosli I finally got back to this!

1. The error messages are a bit buggy still

I couldn't reproduce this error, do you have any more details?

2. The resetting is broken now, it resets to an empty form, instead of what is currently in database Fixed!

3. Can you remind me how a diagnosis gets confirmed again? I'm not able to get one confirmed (this might be a bug?) The diagnosis is confirmed when the visit is set to "Send to DCC".

[driusan]

I'm not sure who's maintaining this PR now but the tests on it are failing and there's a few eslint warnings that need to be fixed (the error ones are the reason it's failing)

/home/runner/work/Loris/Loris/modules/configuration/jsx/DiagnosisEvolution.js
  125:1   error    The type 'int' is undefined                                           jsdoc/no-undefined-types
  303:1   error    The type 'bool' is undefined                                          jsdoc/no-undefined-types
  399:13  warning  Unexpected console statement                                          no-console
  494:13  warning  Unexpected console statement                                          no-console
  504:1   warning  @param "id" does not match an existing function parameter             jsdoc/check-param-names
  548:1   warning  @param "pendingValKey" does not match an existing function parameter  jsdoc/check-param-names

[zaliqarosli]

@driusan ya im working on it

[driusan]

I am getting 2 errors and a blank page on the configuration -> Diagnosis Evolution page:

Warning: ReactDOM.render is no longer supported in React 18. Use createRoot instead. Until you switch to the new API, your app will behave as if it's running React 17. Learn more: https://reactjs.org/link/switch-to-createroot [react-dom.development.js:10:1247](http://localhost:8000/vendor/js/react/react-dom.development.js)

Uncaught ReferenceError: FormElement is not defined
    renderDiagnosisForm DiagnosisEvolution.js:158
    render DiagnosisEvolution.js:291
    React 16
    componentDidMount DiagnosisEvolution.js:54
    promise callback*componentDidMount DiagnosisEvolution.js:54
    React 12
    <anonymous> DiagnosisEvolution.js:582
    EventListener.handleEvent* DiagnosisEvolution.js:581
    <anonymous> DiagnosisEvolution.js:6
    <anonymous> DiagnosisEvolution.js:6
[DiagnosisEvolution.js:158:20](webpack://lorisjs.configuration/modules/configuration/jsx/DiagnosisEvolution.js)
    renderDiagnosisForm DiagnosisEvolution.js:158
    render DiagnosisEvolution.js:291
    React 11
    performSyncWorkOnRoot self-hosted:1406
    React 5
    componentDidMount DiagnosisEvolution.js:54
    (Async: promise callback)
    componentDidMount DiagnosisEvolution.js:54
    React 7
    performSyncWorkOnRoot self-hosted:1406
    React 5
    <anonymous> DiagnosisEvolution.js:582
    (Async: EventListener.handleEvent)
    <anonymous> DiagnosisEvolution.js:581
    <anonymous> DiagnosisEvolution.js:6
    <anonymous> DiagnosisEvolution.js:6

The first seems to be because something is written using an old/deprecated style from react. The second is because import statements from jsx/Form are missing (I'm getting similar errors on the candidate_parameters page but I'll fix those on the other tabs and send a PR since I think that's broken on main.)

[driusan]

Actually it looks like the imports are already there on main. The branch needs to be rebased so that the candidate_parameters loads (and the new tab may need to be fixed.)

[zaliqarosli]

@driusan fixed, and checks passed. could you have another look? @CamilleBeau would you be able to do a final quick run through test? i had to make some extra changes after the rebase (see last 4 commits)

[zaliqarosli]

@driusan the SearchableDropdown being used is just the one that is in the jsx form elements file so its UI can be targeted in a separate if necessary. I added the FK for test name, ready for your review/testing again