android-fhir
android-fhir copied to clipboard
Adds Benchmark module to evaluate the SDK's first run performance
Fixes #1474, #825 and #1378, potential merger with #1414
Description Adds a simple benchmark module to run performance tests on the engine and workflow. The module is focused on the first-time runs, which drive most of the performance issues we are having. The tests must be run in order.
I am not convinced that yet another module needs to be added, but given our struggles with performance, it might be a good short-term investment.
Current results in nanoseconds (seconds,milli,micro,nano) on an older Moto Z Play XT1635-02: (Android8, 3GB RAM, Octa-core 2.0 GHz) with Geekbench: 147 single core and 544 multi-core. These are single runs, but first-time runs (no caching):
49,577,870 ns trace AJacksonMapperBenchmark.loadJsonMapper
2,258,560,068 ns trace BFhirContextLoaderBenchmark.loadR4
2,642,449,743 ns trace BFhirContextLoaderBenchmark.loadR5
1,665,396,000 ns trace BFhirContextLoaderBenchmark.loadDstu3
6,920,873,294 ns trace CCqlEngineFhirContextLoaderBenchmark.loadDstu2FhirModelResolver
3,897,679,346 ns trace CCqlEngineFhirContextLoaderBenchmark.loadR4FhirModelResolver
2,907,996,227 ns trace CCqlEngineFhirContextLoaderBenchmark.loadDstu3FhirModelResolver
630,341,938 ns trace DFhirJsonParserBenchmark.parseLightFhirLibrary
1,258,804,813 ns trace DFhirJsonParserBenchmark.parseLightFhirBundle
2,422,628,522 ns trace EElmJsonLibraryLoaderBenchmark.parseImmunicyCheckCqlFromFhirLibrary
2,481,479,986 ns trace EElmJsonLibraryLoaderBenchmark.parseFhirHelpersCqlFromFhirLibrary
116,937,355 ns trace FEngineDatabaseBenchmark.createAndGet
8,807,674,783 ns trace GCqlEvaluatorBenchmark.evaluatesLibrary (evaluation only)
Pixel 6 GB7N6 (Android 12, 8GB RAM, Octa-core (2x2.80 GHz Cortex-X1 & 2x2.25 GHz Cortex-A76 & 4x1.80 GHz Cortex-A55) with Geekbench: 1036 single core and 2720 multi-core:
21,242,350 ns trace AJacksonMapperBenchmark.loadJsonMapper
335,387,573 ns trace BFhirContextLoaderBenchmark.loadR4
356,097,657 ns trace BFhirContextLoaderBenchmark.loadR5
271,354,411 ns trace BFhirContextLoaderBenchmark.loadDstu3
693,789,348 ns trace CCqlEngineFhirContextLoaderBenchmark.loadDstu2FhirModelResolver
500,146,485 ns trace CCqlEngineFhirContextLoaderBenchmark.loadR4FhirModelResolver
366,984,538 ns trace CCqlEngineFhirContextLoaderBenchmark.loadDstu3FhirModelResolver
152,651,122 ns trace DFhirJsonParserBenchmark.parseLightFhirLibrary
208,492,962 ns trace DFhirJsonParserBenchmark.parseLightFhirBundle
297,588,461 ns trace EElmJsonLibraryLoaderBenchmark.parseImmunicyCheckCqlFromFhirLibrary
276,356,160 ns trace EElmJsonLibraryLoaderBenchmark.parseFhirHelpersCqlFromFhirLibrary
32,813,435 ns trace FEngineDatabaseBenchmark.createAndGet
1,018,112,914 ns trace GCqlEvaluatorBenchmark.evaluatesLibrary (evaluation only)
Alternative(s) considered I tried building an app and a macro module, but I think this micromodule is a better approach.
Type Testing
Screenshots (if applicable)
Checklist
- [x] I have read and acknowledged the Code of conduct.
- [x] I have read the Contributing page.
- [x] I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA.
- [x] I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
- [x] I have run
./gradlew spotlessApply
and./gradlew spotlessCheck
to check my code follows the style guide of this project. - [x] I have run
./gradlew check
and./gradlew connectedCheck
to test my changes locally. - [x] I have built and run the demo app(s) to verify my change fixes the issue and/or does not break the demo app(s).
Codecov Report
Merging #1489 (d042214) into master (5f8790d) will increase coverage by
42.34%
. The diff coverage isn/a
.
@@ Coverage Diff @@
## master #1489 +/- ##
=============================================
+ Coverage 0 42.34% +42.34%
- Complexity 0 384 +384
=============================================
Files 0 150 +150
Lines 0 5405 +5405
Branches 0 971 +971
=============================================
+ Hits 0 2289 +2289
- Misses 0 2758 +2758
- Partials 0 358 +358
Impacted Files | Coverage Δ | |
---|---|---|
.../android/fhir/sync/upload/LocalChangesPaginator.kt | 0.00% <0.00%> (ø) |
|
...alidation/PrimitiveTypeAnswerMaxLengthValidator.kt | 22.22% <0.00%> (ø) |
|
...d/fhir/datacapture/utilities/TypeConversionUtil.kt | 0.00% <0.00%> (ø) |
|
...droid/fhir/db/impl/entities/PositionIndexEntity.kt | 0.00% <0.00%> (ø) |
|
...gle/android/fhir/db/impl/SQLCipherSupportHelper.kt | 71.42% <0.00%> (ø) |
|
...android/fhir/datacapture/QuestionnaireViewModel.kt | 47.55% <0.00%> (ø) |
|
...a/com/google/android/fhir/index/ResourceIndexer.kt | 49.75% <0.00%> (ø) |
|
...oid/fhir/workflow/FhirEngineTerminologyProvider.kt | 0.00% <0.00%> (ø) |
|
...r/datacapture/views/QuestionnaireItemHeaderView.kt | 82.35% <0.00%> (ø) |
|
...apture/views/QuestionnaireItemViewHolderFactory.kt | 100.00% <0.00%> (ø) |
|
... and 140 more |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
Might close or enhance #1492 @Tarun-Bhardwaj
@vitorpamplona Do you have any ETA for the changes?