pynps icon indicating copy to clipboard operation
pynps copied to clipboard

[Enhancement] Multithread download

Open smvueno opened this issue 5 years ago • 8 comments
trafficstars

I just love pynps so badly! Really great work!!! By using this for a day now and tinkering a bit I thought I didn't want to clutter the other PS3 Enhancement request thread too much. So I made this new wishlist here instead!

These are the main 2 things I would love to see.

  1. Automatic download of all found matches, without writing what to download or pressing y.
  2. Multithread download (This would be soo sweet to use on a 1gbit network)

smvueno avatar Jun 08 '20 17:06 smvueno

Hi, thank you for your requests! About 1, this should be easy to do, I'll just add a --no-confirm flag for this, I'll be looking into it soon; About 2, I already responded about it in the other issue, but since we are here and talking about it, what do you actually have in mind for doing something like this? About 3, unfortunately that's not possible with wget, I think curl can actually do that, so I would probably need to migrate all my code to use curl. I have to actually test how Sony's servers respond to that. Well, it'll go into my list, but don't expect this to be implemented any time soon since it's a lot of work.

evertonstz avatar Jun 08 '20 18:06 evertonstz

Thank you very much for at least considering my ideas! No stressing, we all have a private life and other priorities too!

About 1'st --no-confirm sounds amazing! Looking forward to it.

Regarding Nr.2 It's defaulting to the root folder on my Linux system and instead of having the DLFOLDER hardcoded. It would be nice to have an --outputdir command to set the {DLFOLDER} and extract folder before download. Being able to choose download location will be better since the games are quite big and the main drive is not always too big on most computers nowdays. Mine is just a 250gb SSD.

Regarding Nr. 3 on the list. Actually if multithread is not possible or a bit difficult, just having multiple games downloaded at the same time would be a great speed up! Currently, I tried by using 3 different console windows on my pc, JPN, USA, EUR separately downloaded and this greatly improved the speed. But it would be nice if it was possible to download multiple games at once. Even if I separate them like this, having 5-10 games downloaded at the same time might be even faster, depending on how the servers respond to this.

smvueno avatar Jun 09 '20 03:06 smvueno

@evertonstz i think aria2 is even better than curl in this case.

framki avatar Jul 02 '20 22:07 framki

@evertonstz i think aria2 is even better than curl in this case.

Aria2 was actually my first choice before wget, it's better than what we have shipped by default on Linux but I think at the time I just didn't found the gap between the two to be that big, specially because I would need yet another dependency outside python. I think for now I'll see what I can get done using curl, but I'm currently taking a look at pySmartDL, seems promissing.

evertonstz avatar Jul 03 '20 14:07 evertonstz

Ok, I'll probably need someone with some python knowledge that has a fast enough internet to implement this or at least give me some help. I only have 120Mbps down and unfortunately I get max speed with wget, aria2 and pySmartDL, so testing for me is not possible :/

I'll keep the issue just in case someone interested into tackling this show up. I'll edit this issue and create a new one for Auto Download.

evertonstz avatar Jul 04 '20 19:07 evertonstz

Hi again @evertonstz ! Have you tried Google colab? https://colab.research.google.com/ You got gigabit download access there and can try the download speeds easily. I am using it for all my testing too.

smvueno avatar Jul 05 '20 00:07 smvueno

Hi again @evertonstz ! Have you tried Google colab? https://colab.research.google.com/ You got gigabit download access there and can try the download speeds easily. I am using it for all my testing too.

Interesting! wasn't aware about this! I'll take look there as soon as possible. Thanks!

EDIT: it seems colab also won't help, I can only run python code there, since my tests also need a Linux system to run aria2 and wget for comparison purposes, testing is impossible.

evertonstz avatar Jul 05 '20 01:07 evertonstz

Auto download implemented at #23

evertonstz avatar Jul 05 '20 16:07 evertonstz