Savestate restore fails with `Bad file descriptor` trying to reallocate `[stack]`

[jakobhellermann]

Save: http://0x0.st/K9tb.txt Restore http://0x0.st/K9tq.txt

[f:1 t:48218M] DEBUG: Region 0x7ffc59594000 ([stack]) with size 8380416 must be deallocated
[f:1 t:48218M] DEBUG: Region 0x7ffc59595000 ([stack]) with size 8376320 must be allocated
[f:1 t:48218M] DEBUG: Restoring non-anonymous area, 8376320 bytes at 0x7ffc59595000 from [stack] + 0
[f:1 t:48218M] FATAL (/home/jakob/dev/contrib/libTAS-cmake/src/library/checkpoint/Checkpoint.cpp:612): Mapping 8376320 bytes at 0x7ffc59595000 failed: Bad file descriptor (flags=AREA_PRIV|AREA_STACK, mmap=MAP_PRIVATE|MAP_FIXED)
[f:1 t:48218M] FATAL (/home/jakob/dev/contrib/libTAS-cmake/src/library/checkpoint/Checkpoint.cpp:616): Area at 0x7ffc59595000 got mmapped to 0xffffffffffffffff

No idea why the stack is moving by a page.

The code does mmap(addr, size, prot, MAP_FIXED|MAP_PRIVATE, fd=-1, offset). https://github.com/clementgallet/libTAS/blob/5288bdeac3c915d12fe85dae3aeabdf200d74d40/src/library/checkpoint/Checkpoint.cpp#L612

If the stack is merely resized and stays at the addr it is special cased to do a mremap.

Hacking a simple if (flags & AREA_STACK) mf |= MAP_ANONYMOUS into Area::toMmapFlag fixes the crash and I get to All threads resumed, but the program still hangs with gdb pointing to stopThisThread waitForAllRestored(current_thread). https://github.com/clementgallet/libTAS/blob/3f4791dc77d0f2586493ac33b6310b9d27b25d71/src/library/checkpoint/SaveStateManager.cpp#L754

The program is the one I usually use for testing (rust, bevy game engine, winit windowing library, x11) and I'm pretty sure it worked before. Other programs like hollowknight also work fine. Maybe something changed in the rust compiler recently? I could bisect.

uname -a
Linux jj 6.17.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 02 Nov 2025 17:27:22 +0000 x86_64 GNU/Linux

rustc --version
rustc 1.93.0-nightly (278a90913 2025-10-28)

[jakobhellermann]

I suppose there are two questions

• why does the stack even grow here?
• why doesn't libtas handle that correctly during restore

[jakobhellermann]

Crash is also fixed by

struct rlimit rl;
rl.rlim_cur = 16 * 1024 * 1024;
setrlimit(RLIMIT_STACK, &rl) != 0);

in Stack::grow.

(I still get the hang)

[clementgallet]

I also noticed the stack growing by one page in several games. It does not grow ever again, so just saving the state again does not crash anymore.

Maybe it is growing because we read the last page of the stack? But I don't know why it is not growing anymore

Edit: maybe using setrlimit inside the save state code?

[clementgallet]

Commit bc4e21b does not actually fix the issue directly, but it should make it not happening anymore. In this commit, we remove the need to allocate the full 8MB stack. Now, the stack can grow between savestates, and it does not try to modify its size.

This assumes that the stack can never shrink, so the savestate stack segment is always included inside the current stack segment. There is still some code to try handling stack shrink, or if the game managed to allocate a memory segment inside the reserved 8MB stack space.

This has a small benefit of savestates being smaller now.