Research freetext view use to inform hosting TCO reduction

[mrjones-plip]

Describe the issue it's been decided that storage is the highest cost for hosting total cost of ownership (TCO). It's suspected that CHT's use of CouchDB freetext views cause a large amount of disk space. But how much? How might the size of the view be reduced? Where does the UI surface the use of these views?

Describe the improvement you'd like research and document use of freetext views so we know how to safely reduce the disk use caused by using them

Describe alternatives you've considered na

---

Next Steps:

• [x] Check Couch/HAProxy logs for instances of the views being used
• [ ] community research
• [ ] Remove views and run e2e tests and see what happens
• [ ] Add logging for the view and manually poke around the app

Open Questions:

• Where are the views being used programmatically?
• What fields are being searched for?
• What are users actually searching for?

---

CouchDB views for freetext searching

• contacts_by_freetext
• contacts_by_type_freetext
• reports_by_freetext

These get queried by the @medic/search shared lib:

• code that generates queries against the 2 contact-related views: shared-libs/search/src/generate-search-requests.js
• code that generates queries against the report-related view: shared-libs/search/src/generate-search-requests.js

These generated queries are then executed one level higher in shared-libs/search/src/search.jsThis is then imported and ran by:

• the contact exporter in the API: api/src/services/export/contact-mapper.js
• the report exporter in the API: api/src/services/export/report-mapper.js
• the search service in the admin app: admin/src/js/services/search.js
  • which is used by this other service to filter through contact types and places types when editing a user in admin/src/js/controllers/edit-user.js
• the search service in the webapp: webapp/src/ts/services/search.service.ts
  • contacts => webapp/src/ts/modules/contacts/contacts.component.ts
  • contacts => webapp/src/ts/services/contact-view-model-generator.service.ts
    • webapp/src/ts/modules/contacts/contacts-deceased.component.ts
    • webapp/src/ts/services/contact-view-model-generator.service.ts
    • [webapp/src/ts/modules/messages/messages-content.component.ts](https://github.com/medic/cht-core/blob/master/webapp/src/ts/modules/messages/messages-content.component.
    • webapp/src/ts/modules/contacts/contacts-content.component.ts
  • contacts => webapp/src/ts/services/target-aggregates.service.ts
    • webapp/src/ts/modules/analytics/analytics-target-aggregates.component.ts
  • contacts => webapp/src/ts/services/select2-search.service.ts
    • webapp/src/ts/modals/edit-report/edit-report.component.ts
    • webapp/src/ts/modals/send-message/send-message.component.ts
  • reports => webapp/src/ts/modules/reports/reports.component.ts
  • reports => webapp/src/ts/modules/reports/reports.component.ts again
  • reports => webapp/src/ts/services/form.service.ts
    • webapp/src/ts/modals/training-cards/training-cards.component.ts
    • webapp/src/ts/modules/contacts/contacts-edit.component.ts
    • webapp/src/ts/modules/contacts/contacts-report.component.ts
    • webapp/src/ts/modules/reports/reports-add.component.ts

---

logs are VERY informative, even showing the keyword freetext !here's couch logs when searching for bbbbbbbbbbbbbbbbbbbbbbbbbb:

[notice] 2024-07-31T14:12:35.171290Z [email protected] <0.16948.0> 8d6a7d7e03 192-168-68-235.local-ip.medicmobile.org 172.20.0.7 medic GET /medic/_design/medic-client/_view/contacts_by_freetext?startkey=%5B%22bbbbbbbbbbbbbbbbbbbbbbbbbb%22%5D&endkey=%5B%22bbbbbbbbbbbbbbbbbbbbbbbbbb%EF%BF%B0%22%5D 200 ok 9

As well, it shows up in HA proxy for online users when searching for aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:

<150>Jul 31 14:11:16 haproxy[12]: 172.20.0.7,couchdb,200,5,0,0,GET,/medic/_design/medic-client/_view/contacts_by_freetext?startkey=%5B%22aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa%22%5D&endkey=%5B%22aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa%EF%BF%B0%22%5D,-,medic,'-',276,5,-,'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:129.0) Gecko/20100101 Firefox/129.0'