CUE4Parse icon indicating copy to clipboard operation
CUE4Parse copied to clipboard

[FR] Support for deserializing 5.1+ maps with WorldPartition

Open Kein opened this issue 1 year ago • 3 comments

UE version: 5.1.1 Example map: https://pomf2.lain.la/f/ddhmwfxw.7z

Currently it fails at IOEntry reader:

OverflowException: Array dimensions exceeded supported range.
This exception was originally thrown at this call stack:
    CUE4Parse.UE4.IO.IoStoreReader.Read(long, long) in IoStoreReader.cs
    CUE4Parse.UE4.IO.IoStoreReader.Extract(CUE4Parse.UE4.VirtualFileSystem.VfsEntry) in IoStoreReader.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.Read() in FIoStoreEntry.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.CreateReader() in FIoStoreEntry.cs
    ```

Kein avatar Jun 17 '23 19:06 Kein

Small update: the generated tiles-maps are deserialized sucessfully. But the root persistent map/WP map itself fails. Case of commercial games: Talos Principle 2

Kein avatar Nov 21 '23 17:11 Kein

This is also happening with Palworld.

cozzbp avatar Jan 27 '24 02:01 cozzbp

The reason for this is because the world ubulk file is more than 2 GB. The asset and export data is usually around 300 MB. The ubulk file is responsible for a lot of the lighting and terrain data.

There needs to be significant refactors to how deserialization is done for large files. Unreal loads these on a chunk-by-chunk basis as they are accessed, CUE4Parse (and by extension every unreal asset processor) loads the entire file into memory.

A quick workaround is to lazy-load the ubulk until needed, or just not load large ubulks at all and silently fail things that need it.

yretenai avatar Feb 06 '24 06:02 yretenai