CUE4Parse
CUE4Parse copied to clipboard
[FR] Support for deserializing 5.1+ maps with WorldPartition
UE version: 5.1.1 Example map: https://pomf2.lain.la/f/ddhmwfxw.7z
Currently it fails at IOEntry reader:
OverflowException: Array dimensions exceeded supported range.
This exception was originally thrown at this call stack:
CUE4Parse.UE4.IO.IoStoreReader.Read(long, long) in IoStoreReader.cs
CUE4Parse.UE4.IO.IoStoreReader.Extract(CUE4Parse.UE4.VirtualFileSystem.VfsEntry) in IoStoreReader.cs
CUE4Parse.UE4.IO.Objects.FIoStoreEntry.Read() in FIoStoreEntry.cs
CUE4Parse.UE4.IO.Objects.FIoStoreEntry.CreateReader() in FIoStoreEntry.cs
```
Small update: the generated tiles-maps are deserialized sucessfully. But the root persistent map/WP map itself fails. Case of commercial games: Talos Principle 2
This is also happening with Palworld.
The reason for this is because the world ubulk file is more than 2 GB. The asset and export data is usually around 300 MB. The ubulk file is responsible for a lot of the lighting and terrain data.
There needs to be significant refactors to how deserialization is done for large files. Unreal loads these on a chunk-by-chunk basis as they are accessed, CUE4Parse (and by extension every unreal asset processor) loads the entire file into memory.
A quick workaround is to lazy-load the ubulk until needed, or just not load large ubulks at all and silently fail things that need it.