oppia-android icon indicating copy to clipboard operation
oppia-android copied to clipboard

Add tests for new build stats utilities

Open BenHenning opened this issue 2 years ago • 3 comments

Is your feature request related to a problem? Please describe. #4092 introduced several new utilities and a new script to help report build stats for PRs. These require unit tests for behavior regression verification.

Describe the solution you'd like Each of the new, untested classes in the PR need tests to cover their main behaviors. Below are the new classes and minimum tests to add (more are likely needed, these are just a first pass):

  • [x] Add tests for Aapt2Client (suggest using the command locally to get an idea on the sort of output that the class will encounter)

    1. testDumpPermissions_nonExistentApk_failsWithError
    2. testDumpPermissions_invalidApk_failsWithError
    3. testDumpPermissions_apkWithNoPermissions_returnsEmptyList
    4. testDumpPermissions_apkWithSomePermissions_returnsListOfQualifiedPermissionNames
    5. testDumpResources_nonExistentApk_failsWithError
    6. testDumpResources_invalidApk_failsWithError
    7. testDumpResources_apkWithNoResources_returnsEmptyList
    8. testDumpResources_apkWithOnlyStrings_returnsListWithResourcesWithTypesAndIds
    9. testDumpResources_apkWithOnlyDrawables_returnsListWithResourcesWithTypesAndIds
    10. testDumpResources_apkWithOnlyLayouts_returnsListWithResourcesWithTypesAndIds
    11. testDumpResources_apkWithManyResources_returnsListWithResourcesWithTypesAndIds
    12. testDumpBadging_nonExistentApk_failsWithError
    13. testDumpBadging_invalidApk_failsWithError
    14. testDumpBadging_apkWithNoExtraBadgingInfo_returnsPackageAndGenericInfo
    15. testDumpBadging_apkWithOnlyUsesFeatures_returnsBadgingInfo
    16. testDumpBadging_apkWithOnlyUsesImpliedFeatures_returnsBadgingInfo
    17. testDumpBadging_apkWithOnlyUsesFeaturesNotRequired_returnsBadgingInfo
    18. testDumpBadging_apkWithOnlyUsesPermission_returnsBadgingInfo
    19. testDumpBadging_apkWithAllOppiaLikeBadgingInfo_returnsBadgingInfo
  • [x] Add tests for ApkAnalyzerClient (suggest using the apkanalyzer command locally to get an idea on the sort of output that the class will encounter since this utility interacts with apkanalyzer through its internal library)

    1. testComputeFileSize_nonExistentApk_failsWithError
    2. testComputeFileSize_invalidApk_failsWithError
    3. testComputeFileSize_apkWithSomeClassesAndResources_returnsCorrectRawApkSize
    4. testComputeFileSize_apkWithMoreClassesAndResources_returnsCorrectRawApkSize
    5. testComputeDownloadSize_nonExistentApk_failsWithError
    6. testComputeDownloadSize_invalidApk_failsWithError
    7. testComputeDownloadSize_apkWithSomeClassesResources_returnsEstDownloadSize
    8. testComputeDownloadSize_apkWithMoreSomeClassesResources_returnsEstDownloadSize
    9. testComputeFeatures_nonExistentApk_failsWithError
    10. testComputeFeatures_invalidApk_failsWithError
    11. testComputeFeatures_apkWithNoFeatures_returnsEmptyList
    12. testComputeFeatures_apkWithSomeFeatures_returnsListOfFeatures
    13. testCompare_firstApk_isNonExistent_failsWithError
    14. testCompare_secondApk_isNonExistent_failsWithError
    15. testCompare_firstApk_isInvalid_failsWithError
    16. testCompare_secondApk_isInvalid_failsWithError
    17. testCompare_complexApks_firstSecondAreSame_returnsEmptyList
    18. testCompare_twoDifferentComplexApks_returnsListOfDifferences
    19. testComputeDexReferencesList_nonExistentApk_failsWithError
    20. testComputeDexReferencesList_invalidApk_failsWithError
    21. testComputeDexReferencesList_apkWithoutDexFiles_returnsEmptyMap
    22. testComputeDexReferencesList_apkWithOneSmallDexFile_returnsMapWithRefCount
    23. testComputeDexReferencesList_apkWithOneLargeDexFile_returnsMapWithRefCount
    24. testComputeDexReferencesList_apkWithMultipleDexFiles_returnsMapWithRefCounts
  • [x] Add tests for BundleToolClient (suggest using the bundletool utility locally to get an idea on the sort of output that the class will encounter--you can download it using the instructions at: https://developer.android.com/tools/bundletool)

    1. testBuildApks_nonExistentInputAab_failsWithError
    2. testBuildApks_invalidInputAab_failsWithError
    3. testBuildApks_aabWithOneConfiguration_returnsListWithSingleApkFile
    4. testBuildApks_aabWithMultipleConfigurations_returnsListWithAllApkFiles
    5. testBuildUniversalApk_nonExistentInputAab_failsWithError
    6. testBuildUniversalApk_invalidInputAab_failsWithError
    7. testBuildUniversalApk_aabWithOneConfiguration_returnsValidUniversalApkFile
    8. testBuildUniversalApk_aabWithMultipleConfigurations_returnsValidUniversalApkFile
  • [ ] Add tests for ComputeAabDifferences (suggest running the script locally on Oppia AABs to get an idea of the kinds of output it can produce in different circumstances)

  1. testMain_noArguments_failsWithError
  2. testMain_oneArgument_failsWithError
  3. testMain_twoArguments_failsWithError
  4. testMain_threeArguments_failsWithError
  5. testMain_fourArguments_failsWithError
  6. testMain_sixArguments_failsWithError
  7. testMain_sevenArguments_failsWithError
  8. testMain_fiveArguments_invalidBeforeAab_failsWithError
  9. testMain_fiveArguments_invalidAfterAab_failsWithError
  10. testMain_eightArguments_invalidBeforeAabForConfig2_failsWithError
  11. testMain_eightArguments_invalidAfterAabForConfig2_failsWithError
  12. testMain_fiveArguments_validAabs_generatesShortAndLongSummariesForOneConfig
  13. testMain_eightArguments_validAabs_generatesShortAndLongSummariesForTwoConfigs
  14. testComputeBuildStats_zeroProfiles_returnsEmptyBuildStats
  15. testComputeBuildStats_oneProfile_invalidBeforeAab_throwsException
  16. testComputeBuildStats_oneProfile_invalidAfterAab_throwsException
  1. testComputeBuildStats_oneProfile_sameAab_returnsCorrectAabStatsWithNoDiffs
  2. testComputeBuildStats_oneProfile_diffAabs_returnsCorrectConfigAndDiffStats
  3. testComputeBuildStats_twoProfiles_sameAabs_returnsCorrectAabStatsWithNoDiffsForEach
  4. testComputeBuildStats_twoProfiles_diffAabs_returnsCorrectConfigAndDiffStatsForEach
  5. testAabStats_writeSummaryTo_emptyStats_printsMinimalOutput
  6. testAabStats_writeSummaryTo_statsAndDiffs_lowItemLimit_reducesOutput
  7. testAabStats_writeSummaryTo_statsAndDiffs_longSummaryOff_reducesOutput
  8. testAabStats_writeSummaryTo_statsAndDiffs_longSummaryOn_withItemLimit_printsLimitedDetails
  9. testAabStats_writeSummaryTo_statsAndDiffs_longSummaryOn_withLimit_printsExtensiveDetails
  10. testEntireScript_usingMain_forOneProfile_generatesCorrectShortSummary
  11. testEntireScript_usingMain_forOneProfile_generatesCorrectLongSummary
  12. testEntireScript_usingMain_forTwoProfiles_generatesCorrectShortSummary
  13. testEntireScript_usingMain_forTwoProfiles_generatesCorrectLongSummary

Describe alternatives you've considered N/A -- All utilities require tests.

Additional context Each of the utilities above already have existing placeholder test suites that are ready for new tests to be added, and should already build, run, and pass.

All other script & script utility tests are good to reference when building these. Note that these tests can only be run with Bazel and syntax highlighting/auto completion won't work in Android Studio unless using the Bazel plugin.

Note also that these tests often require creating real test data (such as APKs) to verify their functionality. This should be done without needing to permanently check in any binary files to the repository (prefer generating the files at runtime during the test run, instead).

BenHenning avatar May 12 '23 21:05 BenHenning

Hi @theMr17, would it be possible for me to work on a section of this issue that you’re not currently working on?

manas-yu avatar Mar 06 '25 12:03 manas-yu

Hey @manas-yu, thanks for checking in. I had initially started with ComputeAabDifferences, but I'm not actively working on it at the moment. Feel free to assign yourself and proceed with the other sections.

theMr17 avatar Mar 07 '25 02:03 theMr17

Remaining tests are the ComputeAabDifferences tests that are NOT greyed out. 1-13.

adhiamboperes avatar Apr 09 '25 08:04 adhiamboperes