go-download-web icon indicating copy to clipboard operation
go-download-web copied to clipboard

Published 20 hours ago • verified publisher icon antsanchez

Parallelized attachment downloading + fixed relative URL bug

Open CalderWhite opened this issue 1 year ago • 5 comments

Also added .csv and .parquet extensions to valid attachments so you can mass download websites hosting data!

CalderWhite avatar Oct 14 '23 02:10 CalderWhite

Thanks for the changes, but you have changed also github.com/antsanchez/ for github.com/CalderWhite

antsanchez avatar Oct 18 '23 06:10 antsanchez

@antsanchez sorry, that was just for personal use while I was waiting on a response. I can remove that if you are interested in merging in the changes!

CalderWhite avatar Oct 23 '23 07:10 CalderWhite

I think there also might be an issue where it never closes all the channels because I often have to ctrl+c when I am done downloading. Haven't gotten around to debugging why I can't gracefully exit (likely some issue with the channels + concurrency)

CalderWhite avatar Oct 23 '23 07:10 CalderWhite

Update on that bug, I just ran it on an ubuntu system and it exited perfectly! I might have fixed the bug and forgotten I did...

CalderWhite avatar Oct 23 '23 21:10 CalderWhite

Hi, thanks again and sorry for the late reply! Can you change the path: github.com/antsanchez/ instead of github.com/CalderWhite so I can merge the PR? About the last fixed bug you commented, it is already included on the PR?

antsanchez avatar Jan 07 '24 10:01 antsanchez