Only download changed/new assets?

[YumYummity]

Do you think it would be possible to make a sssekai method to only download changed/new assets? Preferably from a downloaded AssetBundleInfo json file, compare differences and download new assets (and save new json).

Then could output a list of changed files?

I'm also thinking of having the regex option still, where it'll only download changes for files matching a regex.

[mos9527]

This is somewhat discussed in #9. Implementation wise this seems doable - we can compare a pair of AbCache cahce files by certain keys - In this case these should be enough, as an extra predicate to filtering assets to download. Maybe something like --download-filter-by-cache <older abcache file> would work?

We still can't tracked what files are already downloaded though since we don't know how the actual hash crc values are derived. But this should generally work on the assumption that the download folder already contains them.

[YumYummity]

In this case these should be enough

Yes, the game compares these hashes. This would be enough to determine changes.

We still can't tracked what files are already downloaded though since we don't know how the actual hash crc values are derived. But this should generally work on the assumption that the download folder already contains them.

I was assuming this was the case. AbCache would need to be updated to contain this info. I'm prepared to redownload everything, it won't take too long.

[mos9527]

Hi, sorry for the wait^^ This has been implemented in https://github.com/mos9527/sssekai/commit/ff6264078bf59f719a530cecd73de3d0df3596dc For some extra context this is how the log looks like with these argumetns

{
    "name": "Python: AbCache (JP)",
    "type": "python",
    "request": "launch",
    "module": "sssekai",
    "args": [
        "--log-level",
        "DEBUG",
        "abcache",
        "--app-region",
        "jp",
        "--app-version",
        "5.1.1",
        "--app-appHash",
        "539e48e0-8f54-8e09-a0b2-1c3ecdb5b178",
        "--download-filter",
        ".*/stage_decoration/.*",
        "--download-dir",
        "~/.sssekai/bundles/",
        "--no-update",
        "--download-filter-cache-diff",
        "~/.sssekai/abcache_old.db",
    ],
    "justMyCode": true
},

With --download-filter-cache-diff

...
15:09:29 Tequila.local sssekai.abcache[51736] DEBUG Skipped live_pv/model/stage_decoration/0129 due to cache diff hit (hash)
15:09:29 Tequila.local sssekai.abcache[51736] DEBUG Skipped live_pv/model/stage_decoration/0500 due to cache diff hit (hash)
15:09:29 Tequila.local sssekai.abcache[51736] DEBUG Skipped live_pv/model/stage_decoration/0508 due to cache diff hit (hash)
15:09:29 Tequila.local sssekai.abcache[51736] DEBUG Skipped live_pv/model/stage_decoration/9527 due to cache diff hit (hash)
15:09:29 Tequila.local sssekai.abcache[51736] INFO Selected 6 bundles to download
15:09:29 Tequila.local sssekai.abcache[51736] INFO Downloading 6 bundles to /Users/mos9527/.sssekai/bundles/

Versus without the flag

...
15:09:59 Tequila.local sssekai.abcache[51793] INFO Filtering bundles with regex pattern: .*/stage_decoration/.*
15:09:59 Tequila.local sssekai.abcache[51793] INFO Selected 175 bundles to download
15:09:59 Tequila.local sssekai.abcache[51793] INFO Downloading 175 bundles to /Users/mos9527/.sssekai/bundles/