derrod
Fix download issue for games with large number of savefiles and improve performance
While testing with a game that has ~650 savefiles, I noticed that Legendary wasn't downloading any data — the download process completed instantly without actually retrieving any files.
Root Cause
Upon investigation, I discovered that the Epic Games Store (EGS) API limits the number of files returned in the metadata JSON to 1000 entries. For games with many files, such as save-heavy ones, essential files like the manifest chunks were missing from the response due to this limit.
Fix
To resolve this, I:
- Modified the download logic to request manifest data separately, bypassing the 1000-file limitation.
- Then used the manifest contents to selectively request only the necessary chunks from the API.
- In my test case, this resulted in ~1600 chunk files to be downloaded.
Enhancements
To improve download performance and reliability:
- Implemented parallel chunk downloads, significantly reducing overall download time (for my example):
- Before: 11 minutes 40 seconds
- After: 1 minute 6 seconds
- Added a retry mechanism for failed chunk requests, as parallel requests occasionally encounter transient errors.
@danez derrod has remarked that this repo currently lacks someone willing to actively maintain it. Given the amount of work you've put into this PR, perhaps you could volunteer?
