Arraymancer icon indicating copy to clipboard operation
Arraymancer copied to clipboard

Add progress bar for datasets downloading and unpacking

Open mratsim opened this issue 5 years ago • 1 comments

IMDB dataset takes a while to download and nothing is printed in the command-line.

A progress bar would be nice.

Documentation: https://nim-lang.org/docs/httpclient.html#progress-reporting

Progress reporting

You may specify a callback procedure to be called during an HTTP request. This callback will be executed every second with information about the progress of the HTTP request.

import asyncdispatch, httpclient

proc onProgressChanged(total, progress, speed: BiggestInt) {.async.} =
  echo("Downloaded ", progress, " of ", total)
  echo("Current rate: ", speed div 1000, "kb/s")

proc asyncProc() {.async.} =
  var client = newAsyncHttpClient()
  client.onProgressChanged = onProgressChanged
  discard await client.getContent("http://speedtest-ams2.digitalocean.com/100mb.test")

waitFor asyncProc()

If you would like to remove the callback simply set it to nil.

  client.onProgressChanged = nil

Warning: The total reported by httpclient may be 0 in some cases.

mratsim avatar Dec 08 '18 10:12 mratsim

Actually it's untar-ing IMDB that is very slow.

mratsim avatar Dec 08 '18 21:12 mratsim