android icon indicating copy to clipboard operation
android copied to clipboard

[WIP] Fix OOM bug in automatic upload queue

Open ThijsRay opened this issue 3 years ago • 2 comments

Background

I recently setup a new Nextcloud instance and wanted to synchronize the contents of my Android phone with it. It contains tens of thousands of photos and videos. When I configured the app to automatically upload these photos and videos to Nextcloud, the app became unresponsive and crashed soon after. Disabling the automatic upload feature for that particular folder also became impossible, as the app became unresponsive as soon as it had started up. Reinstalling the app was the only way to use it again, as that would reset the automatic upload settings.

Fix

Exception in thread "Thread-3" java.lang.OutOfMemoryError: Failed to allocate a 48 byte allocation with 1003528 free bytes and 980KB until OOM, target footprint 268435456, growth limit 268435456; giving up on allocation because <1% of heap free after GC.
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
    at java.lang.StringBuilder.<init>(StringBuilder.java:90)
    at sun.security.x509.AVA.toRFC2253CanonicalString(AVA.java:957)
    at sun.security.x509.RDN.toRFC2253StringInternal(RDN.java:444)
    at sun.security.x509.RDN.toRFC2253String(RDN.java:424)
    at sun.security.x509.X500Name.getRFC2253CanonicalName(X500Name.java:730)
    at sun.security.x509.X500Name.hashCode(X500Name.java:383)
    at javax.security.auth.x500.X500Principal.hashCode(X500Principal.java:487)
    at java.util.HashMap.hash(HashMap.java:338)
    at java.util.LinkedHashMap.get(LinkedHashMap.java:464)
    at okhttp3.internal.tls.BasicTrustRootIndex.<init>(BasicTrustRootIndex.kt:57)
    at okhttp3.internal.platform.Platform.buildTrustRootIndex(Platform.kt:163)
    at okhttp3.internal.platform.Platform.buildCertificateChainCleaner(Platform.kt:160)
    at okhttp3.internal.platform.Android10Platform.buildCertificateChainCleaner(Android10Platform.kt:82)
    at okhttp3.internal.tls.CertificateChainCleaner$Companion.get(CertificateChainCleaner.kt:42)
    at okhttp3.OkHttpClient$Builder.sslSocketFactory(OkHttpClient.kt:870)
    at com.nextcloud.common.PlainClient$Companion.createDefaultClient(PlainClient.kt:62)
    at com.nextcloud.common.PlainClient$Companion.access$createDefaultClient(PlainClient.kt:47)
    at com.nextcloud.common.PlainClient.<init>(PlainClient.kt:45)
    at com.nextcloud.client.network.ClientFactoryImpl.createPlainClient(ClientFactoryImpl.java:94)
    at com.nextcloud.client.network.ConnectivityServiceImpl.isInternetWalled(ConnectivityServiceImpl.java:79)
    at com.owncloud.android.files.services.FileUploader.retryFailedUploads(FileUploader.java:1093)
    at com.owncloud.android.utils.FilesSyncHelper.lambda$restartJobsIfNeeded$0(FilesSyncHelper.java:240)
    at com.owncloud.android.utils.FilesSyncHelper$$ExternalSyntheticLambda0.run(Unknown Source:10)
    at java.lang.Thread.run(Thread.java:920)

The stacktrace listed above pointed me in the right direction. The thousands of files are queued up to be uploaded to the Nextcloud server, an out-of-memory error occurs on my device (6GB of RAM). This pull request prevents this OOM because it limits the amount of files that are added to the upload queue to a, somewhat arbitrary, 250 files at a time. This fixes the issue for me. Others seem to have reported having similar issues:

  • https://github.com/nextcloud/android/issues/10707#issuecomment-1236186431
  • https://github.com/nextcloud/android/issues/9639
  • https://github.com/nextcloud/android/issues/10653
  • https://github.com/nextcloud/android/issues/10633
  • https://github.com/nextcloud/android/issues/10682

ThijsRay avatar Sep 12 '22 13:09 ThijsRay

Unfortunately, this code only fixes the issue temporarily. The upload queue still grows more than it should, albeit more slowly. I will look into this later and try to find a better solution.

ThijsRay avatar Sep 13 '22 07:09 ThijsRay

master-IT test failed, but no output was generated. Maybe a preliminary stage failed.

nextcloud-android-bot avatar Sep 13 '22 10:09 nextcloud-android-bot

Thanks! Hope this gets merged or completely fixed soon.

Seb3thehacker avatar Oct 21 '22 09:10 Seb3thehacker

Looks like @AlvaroBrey has fixed this issue 5 days ago with https://github.com/nextcloud/android/commit/e4a922b913fed661888c40f5b6280cb17584285c :tada:

ThijsRay avatar Nov 02 '22 19:11 ThijsRay

Looks like @AlvaroBrey has fixed this issue 5 days ago with e4a922b tada

For what it's worth, this will only fix it for now in Android 12+ or beta/dev versions. We will roll it out to more Android versions gradually

AlvaroBrey avatar Nov 02 '22 21:11 AlvaroBrey