feat(#9869): add storageFree and storageTotal to user-devices API

[najuna-brian]

Description

This PR continues the work from #9897, which was closed after a force push. Closes #9869

cc @dianabarsan @mrjones-plip

Summary

This pull request enhances the /api/v2/export/user-devices endpoint by adding storageFree and storageTotal fields for each user device. This addresses the need for administrators to easily monitor device disk space, which can help in diagnosing performance issues related to low storage on Community Health Worker (CHW) devices.

Previously, obtaining this information was a hard process involving manual checks of telemetry documents in CouchDB or setting up complex data synchronization pipelines.

Implementation Details

The following changes were made to implement this feature:

1. CouchDB View Updated (ddocs/users-meta-db/users-meta/views/device_by_user/map.js):

   • The map function was modified to extract doc.device.deviceInfo.storage.free and doc.device.deviceInfo.storage.total from telemetry documents.
   • These values are now emitted as part of the device information.

2. API Service Modified (api/src/services/export/user-devices.js):

   • The service now retrieves the storage.free and storage.total values from the data emitted by the CouchDB view.
   • These are included in the final JSON response as storageFree and storageTotal respectively.

3. Tests Updated:

   • Unit Tests (api/tests/mocha/services/export-data.spec.js): Updated to include mock data and assertions for the new storageFree and storageTotal fields in the user-devices export.
   • Integration Tests (tests/integration/api/controllers/export-data.spec.js):
     • The createTelemetryDoc helper was updated to allow inclusion of storage information.
     • Tests that interact with the /api/v2/export/user-devices endpoint were updated to verify the presence and correctness of the new storage fields in the API response.

How to Test

Please follow the detailed testing steps outlined in issue #9869. Key verification points include:

1. Setting up a development environment and creating a test user with telemetry data containing storage information (as per the sample JSON in the issue).
2. Making a curl call to http://medic:password@localhost:5988/api/v2/export/user-devices.
3. Verifying that the output for the test user's device now includes the storageFree and storageTotal fields with the correct values from the telemetry document.

Affected Files

• api/src/services/export/user-devices.js
• api/tests/mocha/services/export-data.spec.js
• ddocs/users-meta-db/users-meta/views/device_by_user/map.js
• tests/integration/api/controllers/export-data.spec.js

Code review checklist

• [ ] UI/UX backwards compatible: Test it works for the new design (enabled by default). And test it works in the old design, enable can_view_old_navigation permission to see the old design. Test it has appropriate design for RTL languages.
• [ ] Readable: Concise, well named, follows the [style guide](https://docs.communityhealthtoolkit.org/contribute/code/style-guide/), documented if necessary.
• [ ] Documented: Configuration and user documentation on [cht-docs](https://github.com/medic/cht-docs/)
• [ ] Tested: Unit and/or e2e where appropriate
• [ ] Internationalised: All user facing text
• [ ] Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

[najuna-brian]

@dianabarsan Thanks for the feedback! I've updated createTelemetryDoc and removed unnecessary null checks. I have also fixed the integration test failures. Please let me know if any further changes are needed!

[najuna-brian]

@dianabarsan Thanks for the review! I've addressed all your feedback:

CouchDB View Optimization: Introduced deviceInfo variable to reduce null checks and improve readability as suggested Browser Property Logic: Reverted the browser property changes to maintain existing behavior and compatibility
Code Integration: Fixed the duplicate emit issue and properly integrated the storage fields into the existing emit block Indentation: Reverted indentation changes to maintain consistency

All tests are passing

Could you please re-review Thanks!

[najuna-brian]

@dianabarsan The CI shows some WebDriver test failures that appear to be flaky infrastructure issues:

• ci-webdriver-default-mobile-all: UI element timing issue in bulk delete test (element still displayed after 15000ms)
• Similar timing issues in integration tests

These failures are unrelated to the storage field changes since:

• All unit tests pass (including new storage field tests)
• Code quality checks pass
• My changes only add API response fields, no UI modifications
• The failures are WebDriver timeout issues, not logic errors

Are these known flaky tests that can be restarted, or should I investigate further? The core functionality is proven working via passing unit tests.

[mrjones-plip]

@najuna-brian - I've kicked off a re-run of failed jobs in case they are indeed flaky

[najuna-brian]

Thanks for retriggering those! @mrjones-plip

[najuna-brian]

@mrjones-plip The retrigger shows ci-webdriver-default-core now passes (was failing)

[mrjones-plip]

Hitting it again in hopes of the remaining 3 passing...

[najuna-brian]

For the last 2 failing tests,

Fixed failing test: createTelemetryDoc was always creating storage objects, causing assertion mismatch (undefined vs [undefined] in view emissions).

Now conditionally creates storage only when values exist - aligns with real device telemetry patterns.

[mrjones-plip]

re-running :crossed_fingers: !

[najuna-brian]

It looks like CouchDB view was always emitting storage even with undefined values, causing the integration test failures. Fixed the CouchDB view to conditionally create storage objects

Now storage is only added when storageFree or storageTotal have actual values.

[najuna-brian]

Hi @mrjones-plip, @dianabarsan I would appreciate approval of the workflow on the newly pushed changes Thank you

[mrjones-plip]

I've merged master to this branch and I'll push the tests along if they get stuck, but otherwise letting the natural review process do its thing.

[najuna-brian]

@mrjones-plip @dianabarsan The failing checks show 502/503 errors which appear to be CI environment issues. Could you please re-run the failing tests? Thank you!

[najuna-brian]

Wow, it took me a whole 5 months 😅 thanks a lot @dianabarsan and @mrjones-plip for the patience and guidance, really appreciate the support!

[mrjones-plip]

Excellent work @najuna-brian ! Great job sticking with this PR to see it through to being merged!