Implement sending remote resource packs to Bedrock clients

[onebeastchris]

Since 1.20.30, it is possible to send clients a link to download a resource pack from - instead of having to add the pack to Geyser's pack folder. This should help with proxy networks & overall traffic usage, as now resource intensive pack sending can be offloaded to a designated location. However, unlike on Java, it is possible to send multiple entries for the client to load packs from! Further, this allows updating the remote resource pack on-the-fly: Geyser will detect the change, and re-download the pack.

[!IMPORTANT]
Note: You'll need to update Bedrock to the latest client version (1.21.1). 1.21.0 is unfortunately bugged, and ignores remote resource packs. 1.20.80 should also support this.

---

New config option:

# A list of links to send to the client to download resource packs from.
# These must be direct links to the resource pack, not a link to a page containing the resource pack.
# If you enter a link here, Geyser will download the resource pack once to check if it's in a valid format.
# See https://wiki.geysermc.org/geyser/packs for more info.
resource-pack-urls:
  # GeyserOptionalPack
  - "https://download.geysermc.org/v2/projects/geyseroptionalpack/versions/latest/builds/latest/downloads/geyseroptionalpack"

This will ensure that all connecting clients will, by default, get the GeyserOptionalPack.

---

You can download builds here:

• Geyser-Bungeecord.jar
• Geyser-Fabric.jar
• Geyser-NeoForge.jar
• Geyser-Spigot.jar
• Geyser-Standalone.jar
• Geyser-Velocity.jar
• Geyser-ViaProxy.jar

---

Technical details:

• A Bedrock client will receive the cdn entries (urls) alongside all other packs. If it fails to download the pack from the url, it'll fall back to requesting the pack from Geyser.
• Geyser downloads all packs once when starting (or uses a cached download) to ensure that the resource pack is valid, and to parse the pack version/uuid/manifest. Further, it caches a fallback pack to use in the case that the url is offline, not reachable, or just denied by the Bedrock client for some reason.
• Bedrock clients are very picky. The requirements for which links do and don't work are not fully known at the moment. What is known:
  • A "content-length" header field must be set to the correct file size. Otherwise, the pack will not be downloaded.
  • The application/zip content type seems to work best for downloading packs. The often used text/html type does not work (rip onedrive links).
  • Redirects seem to be followed correctly, but the link should still be a direct download
  • The pack can be either a .zip or .mcpack

---

Changes:

• add new GeyserUrlPackCodec (and UrlPackCodec for api) classes that can create a pack based on url + content key
• add a resource-pack-urls entry in the config for remote packs
• try and use more fastutil for resource packs

---

TODO:

• [x] Proper error handling: URL invalid/offline, corrupt pack, manifest not where it's supposed to be, etc.
• [ ] documentation on wiki
• [x] remove debug
• [x] dont let resource pack downloading block main thread - this is actually wanted; since Geyser should not start before we know all the resource packs
• [x] Don't re-download the pack if there's no need to (hash from url + size)
• [x] Testing

Would close https://github.com/GeyserMC/Geyser/issues/4060 - atleast the resource pack part.

[Camotoy]

In the config I would use, probably, resource-pack-urls or remote-resource-packs. I don't think most people will understand what a CDN is, and some resource packs won't be hosted on content delivery networks per se.

[onebeastchris]

Yeah naming really isn't my strong suit, I agree. I kind of like RemotePack actually. Or we could take a page from kyori's adventure, their new resource pack method takes in a ResourcePackRequest (Like).. not sure I really like that though. Open for suggestions :)

[Kas-tle]

How exactly are pack versions handled with this? Is it just pulled from the downloaded pack? What happens if the version of the pack at the link is updated while the server is running?

[onebeastchris]

We don't extract / need the pack version, just the pack UUID. Konica and I had a brief talk where he mentioned that possibility where a pack UUID might change mid-game. Since we can't really detect that, the next best thing would be to use some kind of cache and re-download the configured packs to re-check the pack ids. This still leaves the question of what to do with api - imo we shouldn't do the same check for those remote packs provided there, since they need to provide the pack UUID themselves.

Possible solutions:

• add a user-facing reload command that would re-gather all remote pack uuids (e.g. /geyser reload packs) or similar; would also close a different issue
• Could expose an api method that downloads the pack, extracts it & provides a UUID version to avoid duplicate code in extensions that also want to check
• Could add a secondary registerRemotePacks/..CdnPacks method to the GeyserDefineResourcePacks only that would allow providing just a url and leave us querying the pack id ourselves.

More generally, we'd need to find out what exactly the client would do in the event that the pack is provided and the one received in said pack mismatch.

[Kas-tle]

Hmm ok but then how does this work? Does this mean the client has to download the entire pack every time to check if the version changed? The normal ResourcePackStack and ResourcePacksInfo packets have a field for version. So does a CDN entry also require an entry in the normal Entry list and this just tells the client where it can download said pack? Or will the client just download a CDN entry and not check the version at all?

I'm mainly wondering how this works if the user just adds an entry to their config for one of these. In this scenario you're saying we never have to know the pack version?

[onebeastchris]

~~By what I can tell, we never have to get/tell the client the pack version.This does raise the question of whether the client will always download the pack...~~ I'll definitely have to do more testing here; I am not fully sure about the exact workings of this feature. In my (brief) testing, the optional pack seems to have been loaded fine.

After some testing:

• We need to send an entry in the ResourcePackInfo packet, even for these cdn packs.
• the "packid" in the cdn entry consists of uuid_version, not just the UUID
• the client falls back to requesting the packs normally/the old way if the download didn't succeed
• We can actually detect the last part, and try and re-load packs. Not sure if we can do that fully automatically, but in theory, should be doable.

[onebeastchris]

From more testing locally, here's what i found: This PR should be functional, but Bedrock is very picky about what sort of packs from a url are accepted/downloaded.

• The link must be a direct link.
• The size must be set properly in the Content-Length header. If size is wrong, or not present (-1), the client will show a bogus, stupidly large size value and wont allow downloading the packs.
• Similarly, the Content-Type header must be set to application/zip - otherwise, the client wont download the resource packs.

Further, the resource pack can be either a .zip or .mcpacks file that does not contain the pack files at its root, but rather a folder with the actual resource pack. (In other words: zip/manifest.json wont work, zip/mypack/manifest.json will....)

If anyone wishes to test this PR, you can set up a working file server using caddy and the following config - please do not use it in production, it doesnt even use https.

http://localhost {
    file_server browse
    route {
        header Content-Type application/zip
        header Content-Length {file.size}
    }
}

It however would ensure correct content type/content lengths being set.

On a positive note, downloading even a 100MB pack is insanely quick!

[Camotoy]

Is it possible we can make a website that tests if a website hosting a pack is working correctly? We can also bundle such functionality in Geyser, but a website means it would be applicable to a wider Bedrock audience.

[onebeastchris]

Said functionality would already be bundled in this PR :) We're basically downloading the pack once to get all the info, and while at it, we check size/content type/manifest location (which, to my knowledge, is what's required). As for creating a separate website - sure, but I think that would be a bit out of scope for this PR, but it's not a bad idea in theory.

[Camotoy]

OK, neat. We should add to the config comment that you should expect your pack to be analyzed/downloaded within Geyser.

[onebeastchris]

Well, we basically have to download it. We need to include the pack in the same resource pack stack packet, and we need to get the UUID/version for it. Another neat thing is that we can also detect if the url doesn't properly work anymore (e.g. pack changed mid download/link offline), and then Geyser would re-test the pack.

[onebeastchris]

Put on hold until we transition to using configurate - otherwise these changes will result in confusion about why packs are loaded that don't seem to be in the config.