CDCgov
Refactor `FHIRBundleHelpers.lookupConditionCodings()` to use metadata.table directly
User Story
As an RS engineer
I want to speed up the FHIRBundleHelpers.lookupConditionCodings() function,
so that the FHIR Convert step can process larger files without hitting the timeout period.
Description/Use Case
In local testing the FHIR Convert process() function accounted for 90%+ of the FHIR convert function. Within the process() function, FHIRBundleHelpers.lookupConditionCodings() accounted for 80%+. The call to caseSensitiveDataRowsMap seems to be what takes the most time with each call taking ~100 milliseconds. That call occurs on each observation. Most messages have multiple observations so the time multiplies quickly.
Dev Notes
Currently the function uses caseSensitiveDataRowsMap in LookupTable.kt to create a sequence from mappingTable.table. I would consider directly filtering on mappingTable.table using tablesaw.
LogMeasuredTime.measureAndLogDurationWithReturnedValue()is useful for measuring processing duration.
Acceptance Criteria
- Duration of
lookupConditionCodings()completes in under 25 milliseconds
Hey team! Please add your planning poker estimate with Zenhub @adegolier @arnejduranovic @brick-green @david-navapbc @jack-h-wang @jalbinson @JFisk42 @mkalish @thetaurean
