Unexpected Download Behaviour

[oliverskawronek]

At first, thank you for this library and the nice interface!

For an app I like download multiple files concurrently, but I don't understand the behaviour of the downloads.

Environment

• implementation 'androidx.tonyodev.fetch2:xfetch2:3.1.6'
• Android 10
• minSdk 25, targetSdk 31
• Manifest:
  • uses permission "android.permission.INTERNET"
  • uses permission "android.permission.WRITE_EXTERNAL_STORAGE"
  • <application ... android:requestLegacyExternalStorage="true"

I try to download two files, one produces a 200 OK the other a 400 Bad Request on the server. For the first one I expect an onCompleted-call and for the last one an onError-call.

Test code

val fetchConfiguration = FetchConfiguration.Builder(this)
    .setAutoRetryMaxAttempts(1)
    .enableAutoStart(false)
    .enableLogging(true)
    .setDownloadConcurrentLimit(10)
    .build()

val fetch = getInstance(fetchConfiguration)
fetch.deleteAll()

fetch.addListener(object : AbstractFetchListener() {
    override fun onCompleted(download: Download) {
        Log.i("DOWNLOAD", "Downloaded completed ${download.url}")
    }

    override fun onError(download: Download, error: Error, throwable: Throwable?) {
        Log.e("DOWNLOAD", "Error ${download.url}, error: $error", throwable)
    }
})

val downloadDir = File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS), "tmp")

val request200OK = Request("https://httpstat.us/200",
    File(downloadDir, "ok.txt").absolutePath)

val badRequest400 = Request("https://httpstat.us/400",
    File(downloadDir, "bad_request.txt").absolutePath)

fetch.enqueue(request200OK, {
    Log.i("DOWNLOAD", "Enqueued ${it.url}")
}) { error: Error? ->
    Log.e("DOWNLOAD", "Failed to enqueue ${request200OK.url}, error: $error")
}

fetch.enqueue(badRequest400, {
    Log.i("DOWNLOAD", "Enqueued ${it.url}")
}) { error: Error? ->
    Log.e("DOWNLOAD", "Failed to enqueue ${badRequest400.url}, error: $error")
}

Log output

2021-09-15 15:10:41.386 6896-7119/org.example.download D/LibGlobalFetchLib: Deleted download DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=0, status=DELETED, error=NONE, networkType=ALL, created=1631710349874, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.389 6896-7119/org.example.download D/LibGlobalFetchLib: Deleted download DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=DELETED, error=REQUEST_NOT_SUCCESSFUL, networkType=ALL, created=1631710349890, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=1, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.389 6896-7119/org.example.download D/LibGlobalFetchLib: Added listener org.example.download.MainActivity$onDownload$1@3a84125
2021-09-15 15:10:41.405 6896-7119/org.example.download D/LibGlobalFetchLib: Enqueued download DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441390, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.410 6896-7119/org.example.download D/LibGlobalFetchLib: Added DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441390, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.411 6896-7119/org.example.download D/LibGlobalFetchLib: Queued DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441390, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1) for download
2021-09-15 15:10:41.416 6896-6896/org.example.download I/DOWNLOAD: Enqueued https://httpstat.us/200
2021-09-15 15:10:41.430 6896-7119/org.example.download D/LibGlobalFetchLib: Enqueued download DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.433 6896-7119/org.example.download D/LibGlobalFetchLib: Added DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.433 6896-7119/org.example.download D/LibGlobalFetchLib: Queued DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1) for download
2021-09-15 15:10:41.437 6896-6896/org.example.download I/DOWNLOAD: Enqueued https://httpstat.us/400
2021-09-15 15:10:41.960 6896-7133/org.example.download D/LibGlobalFetchLib: DownloadManager starting download DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:41.971 6896-7132/org.example.download D/LibGlobalFetchLib: DownloadManager starting download DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441390, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:42.915 6896-7132/org.example.download D/LibGlobalFetchLib: FileDownloader starting Download DownloadInfo(id=-1230077665, namespace='LibGlobalFetchLib', url='https://httpstat.us/200', file='/storage/emulated/0/Download/tmp/ok.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=0, status=QUEUED, error=NONE, networkType=ALL, created=1631711441390, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:42.922 6896-7133/org.example.download E/LibGlobalFetchLib: FileDownloader download:DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=0, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
    com.tonyodev.fetch2.exception.FetchException: request_not_successful
        at com.tonyodev.fetch2.downloader.SequentialFileDownloaderImpl.run(SequentialFileDownloaderImpl.kt:129)
        at com.tonyodev.fetch2.downloader.DownloadManagerImpl$start$$inlined$synchronized$lambda$1.run(DownloadManagerImpl.kt:103)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)
2021-09-15 15:10:48.472 6896-7140/org.example.download D/LibGlobalFetchLib: DownloadManager starting download DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=1, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
2021-09-15 15:10:48.799 6896-7140/org.example.download E/LibGlobalFetchLib: FileDownloader download:DownloadInfo(id=44834860, namespace='LibGlobalFetchLib', url='https://httpstat.us/400', file='/storage/emulated/0/Download/tmp/bad_request.txt', group=0, priority=NORMAL, headers={}, downloaded=0, total=-1, status=QUEUED, error=NONE, networkType=ALL, created=1631711441412, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={}, autoRetryMaxAttempts=0, autoRetryAttempts=1, etaInMilliSeconds=-1, downloadedBytesPerSecond=-1)
    com.tonyodev.fetch2.exception.FetchException: request_not_successful
        at com.tonyodev.fetch2.downloader.SequentialFileDownloaderImpl.run(SequentialFileDownloaderImpl.kt:129)
        at com.tonyodev.fetch2.downloader.DownloadManagerImpl$start$$inlined$synchronized$lambda$1.run(DownloadManagerImpl.kt:103)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)
2021-09-15 15:10:53.829 6896-6896/org.example.download E/DOWNLOAD: Error https://httpstat.us/400, error: REQUEST_NOT_SUCCESSFUL
    com.tonyodev.fetch2.exception.FetchException: request_not_successful
        at com.tonyodev.fetch2.downloader.SequentialFileDownloaderImpl.run(SequentialFileDownloaderImpl.kt:129)
        at com.tonyodev.fetch2.downloader.DownloadManagerImpl$start$$inlined$synchronized$lambda$1.run(DownloadManagerImpl.kt:103)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)

I wonder

• SequentialFileDownloaderImpl is part of the logging. I tought setDownloadConcurrentLimit activates something parallel
• Both files ok.txt and bad_request.txt are created and empty
• My expectation is, when an error happens no file is created. Large files should be downloaded in a system temporary directory and moved to the right target after success or get deleted on error.
• For the request to https://httpstat.us/200 I didn't get an onCompleted- or an onError-call
• Both requests remains in the local database. Please see my attached screenshot of the Database Inspector. Shouldn't at least the bad request removed from the database, because it is finished with an error?