Wrong output when number of dex files increases

[lwasyl]

This is output for diffuse when comparing APK before and after adding a big library with native libs and whatnot (unique numbers are way off because of R8):

          │             compressed             │           uncompressed            
          ├───────────┬───────────┬────────────┼───────────┬───────────┬───────────
 APK      │ old       │ new       │ diff       │ old       │ new       │ diff      
──────────┼───────────┼───────────┼────────────┼───────────┼───────────┼───────────
      dex │   2.6 MiB │     196 B │   -2.6 MiB │   6.6 MiB │      -2 B │  -6.6 MiB 
     arsc │ 440.6 KiB │ 439.6 KiB │     -970 B │ 440.5 KiB │ 439.5 KiB │    -960 B 
 manifest │     3 KiB │     113 B │   -2.9 KiB │  12.2 KiB │      -1 B │ -12.3 KiB 
      res │   1.5 MiB │   1.1 MiB │ -496.3 KiB │   2.1 MiB │ 906.7 KiB │  -1.3 MiB 
   native │  43.8 KiB │   4.6 MiB │   +4.6 MiB │    43 KiB │   4.6 MiB │  +4.5 MiB 
    asset │    49 KiB │     421 B │  -48.6 KiB │ 124.9 KiB │      -3 B │  -125 KiB 
    other │ 170.8 KiB │ 112.4 KiB │  -58.4 KiB │ 361.6 KiB │ 275.6 KiB │ -86.1 KiB 
──────────┼───────────┼───────────┼────────────┼───────────┼───────────┼───────────
    total │   4.8 MiB │   6.2 MiB │   +1.4 MiB │   9.7 MiB │   6.1 MiB │  -3.6 MiB 


         │          raw          │                unique                 
         ├───────┬───────┬───────┼───────┬───────┬───────────────────────
 DEX     │ old   │ new   │ diff  │ old   │ new   │ diff                  
─────────┼───────┼───────┼───────┼───────┼───────┼───────────────────────
   count │     1 │     2 │    +1 │       │       │                       
 strings │ 35815 │ 38915 │ +3100 │ 35815 │ 37708 │ +1893 (+5632 -3739)   
   types │ 11415 │ 12329 │  +914 │ 11415 │ 12031 │  +616 (+4007 -3391)   
 classes │ 10163 │ 10693 │  +530 │ 10163 │ 10693 │  +530 (+3701 -3171)   
 methods │ 42939 │ 46614 │ +3675 │ 42939 │ 45943 │ +3004 (+30436 -27432) 
  fields │ 57046 │ 57083 │   +37 │ 57046 │ 57018 │   -28 (+48758 -48786) 

Note DEX#count row, which shows increased DEX count, yet APK#dex row shows dex size has significantly decreased. Also total compressed file size is much lower than in reality. Here's what AS shows when inspecting new apk file:

For comparison, old apk seems to be aligned with diffuse output:

[JakeWharton]

Yeah something is definitely wrong there, it's clearly not attributing space correctly in the second APK. How are you creating the APK? Are you using an external tool designed for obfuscation?

You can tell it still sees the dex files, but their size is returning 1B somehow (hence the 2B total), potentially due to a corrupted zip structure (at least in the eyes of the JDK ZIP implementation). The 196B compressed size is all the ZIP metadata like the file name, path, timestamps, etc. being added to 2B.

Are you able to share the second APK? Or the tools you're using so I can try to reproduce?

Also what version of Java are you running diffuse with?

[lwasyl]

How are you creating the APK? Are you using an external tool designed for obfuscation?

Just plain ./gradlew app:assembleMyVariantRelease with minifyEnabled true. However the new build had FullStory integration enabled (https://help.fullstory.com/hc/en-us/articles/360040596093-Getting-Started-with-Android-Recording) which might be doing some stuff on the apk/dex after the app has been built. I can't share the apk unsurprisingly, but FullStory integration is pretty straightforward (the only tricky part is that root project needs to have AGP in classpath)

Everything is running the same version of Java:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.252-b09, mixed mode)

[shamilovstas]

I have a similar issue when a build is signed using jarsigner (100% reproducibility). Here are the steps to reproduce:

1. Check that an apk isn't signed and run diffuse info on it: The report shows that the build's size is computed correctly.
2. Sign the build with jarsigner and run diffuse info once again: Now the report says that the uncompressed size is -1

Also, the Wikipedia page about ZIP format that you provided in the comments denotes that the local header of a file stored in a zip archive contains information about the file's compressed and uncompressed size only in case if the file's format is not ZIP64: Can it be the source of the issue?

[mateuszkwiecinski]

I don't fully understand why, but I just learned that negative values (both for size and compressed size) are produced only when using ZipInputStream. If I replace ZipInputStream with ZipFile (as per this SO answer suggested) diffuse starts showing expected values.