OpenJK
OpenJK copied to clipboard
JACoders
Serialization for Save games
Save games should be serialized if possible to fix portability issues with saves. Ideally would want to avoid changes to existing structures like gentity etc. Not sure if its possible to retain the format though. Serialization doesn't seem to fit well with how they currently handle things with string limitations and also chunk ids. Also not sure how to serialize across dll boundaries since sv_savegame and g_savegame are both needed to handle the save contents. And avoiding C++ objects across the dll boundary is required.
@mrwonko
Ugh, this sounds like a lot of work.
Compat with existing Windows saves should be possible though, if I'm not mistaken. We "just" need to write a function that manually serializes/deserializes all the fields.
I don't see how that works with chunkids though. Or how to serialize character arrays. Since they seem to just store the max size of the array in bytes but only fill in whats used.
Maybe. But if possible, no new format.
If a new format is used, it should be thought through thoroughly first; it should be future proof.
Should saves be something that needs to be done prior to a release? I kind of think so.
Not sure if the new code qualifies as serialization or maybe it does but now that #878 is done can this be closed?
bb0b2f0a541117aba77fc7b59ee51defda99a5b6
Not sure if the new code qualifies as serialization
I think it does, but one issue that still remains is that it writes numbers in the CPU's native endianness. So you can now have savegames that are portable between 32-bit and 64-bit x86 (which are both little-endian: 0x12345678 is serialized as "\x78\x56\x34\x12"), but not between 32-bit x86 and 32-bit PowerPC (which is big-endian: 0x12345678 is serialized as "\x12\x34\x56\x78").
Because savegames seem to start with a 32-bit "magic number", it would be possible to check whether the magic number is in native or opposite-of-native endianness, and set a flag in the reader object; every time a multi-byte number is read, the reader object would have to byteswap it, and every time an array of numbers is read, the reader object would have to byteswap each member.
Loading ordinary integers etc. (for example array lengths and the FPSL chunk) should be something the template code can do. I noticed while fixing #902 that the Icarus stuff saves/loads its own formatted binary blob, which won't be affected by generic endianness-swapping - but code/icarus/BlockStream.cpp uses LittleLong() so it looks as though someone already thought about endianness there. Any other arbitrary binary blobs will need special care.
Do any of the OpenJK developers have access to a big-endian test machine? A PowerPC (pre-x86) Mac running Linux is probably the most likely. I have one, but won't be able to access it for a while.
You'll need #886 to be able to compile on big-endian machines, assuming you're using a recent version of gcc or clang.