Deltarune desync problem (again)

[BenLubar]

This one doesn't seem to be related to save states. Something is causing the first fight with three Jigsawry to desync at random. Running the game again with the same movie generally "fixes" the issue, but I feel like there's either a race condition somewhere or Deltarune is accessing the clock or something like that before libTAS replaces the functions.

There does not seem to be any similar problem in Undertale, at least as far as I have reached in the game.

[BenLubar]

Ok, this appears to have been due to Deltarune calling RNG in an alternate thread and me having the clock settings wrong for that.

[BenLubar]

Nope, that doesn't seem to have been the root cause. It desynced again on the Jigsawry fight.

The two attacks I've seen desync are Jigsawry's only attack and the attack Hathy uses when she is alone.

[BenLubar]

I am confused at this point - maybe the "correct" version of results for the button presses I'm entering is a desync and I was just getting lucky with how it broke. It seems to be breaking the same way 100% of the time now that I have the clock settings correct.

[BenLubar]

Sorry about closing and re-opening this issue so many times.

I just went in to fix the desync... and it didn't desync. Unless there's something weird going on with fast forward causing something that calls RNG to not be called, I have no idea what could be causing it to desync versus not.

[BenLubar]

Ok, on the left is a full recording of the TAS with the first ten minutes chopped off. On the right is a recording of the TAS after fast forwarding to the ten minute mark without recording.

https://files.lubar.me/deltarune-desync.mp4

[clementgallet]

I don't see where the desync could come from. Is DR using libc rand functions ? If so, did you print random log to see if there is a shift in calls ?

If fast-forward is involved, you could try to force all frames to render, to see if it changes anything ? (you have to modify the code, but it's pretty straightforward, returns false in skipDraw() in library/frame.cpp).

Do you have a movie file so that I can look at it ?

[BenLubar]

I've invited you to the GitLab repo, which is the same as https://github.com/BenLubar/tobytas but also contains the LTM contents.

The desync happens between frames 20175 and 20250.

The two functions it calls for RNG are choose and random. I'm not sure how they're implemented. I tried using the random_get_seed function on every frame, but it always returns 0, so presumably it only knows about the initial seed.

[BenLubar]

Ok, I've done some research:

The random number generator seems to be custom-built. It's located at 00413f30 in the version of runner that the Deltarune for Linux Reddit post includes.

There's a single 32-bit integer argument at $rax-4.

[BenLubar]

Fast forward with all frames rendered does give the "correct" results, but I'm not sure whether that's because it slows down the game logic enough that the race condition doesn't happen or that the rendering code is somehow affecting RNG.

[BenLubar]

The last synchronized RNG call happens on frame 16062, and the first call that only happens on the non-fast-forward version is on frame 16065.

Here's the diff of frames where RNG is called: https://gist.github.com/BenLubar/698870e15e94f98105c7308d323d97d2

A bunch of new stuff appears around that frame:

• The only "now playing" notification in the entire game
• Enemies that chase you in the overworld
• The grass is animated on that screen but not anywhere before then

Since there are a lot of RNG calls that the fast forward gains in addition to the (more) calls that it skips, I'm guessing this isn't due to rendering calling RNG but rather due to some other thread being starved of resources and not getting to call RNG as much as it normally would be able to.

Here's a video of the section of the game where the RNG desyncs: https://files.lubar.me/deltarune-desync-a.mp4

(Coincidentally, 16065 is a non-rendered frame during a room transition.)

[BenLubar]

Alright, to re-synchronize the TAS on frame 20000, this (very long) gdb command can set the RNG to the correct state:

sudo gdb -iex 'set pagination off' -p `pgrep deltarune` -ex 'p *(uint32_t*)0xb459e0 = 0x54d893f4' -ex 'p *(uint32_t*)0xb459e4 = 0x78a5607f' -ex 'p *(uint32_t*)0xb459e8 = 0x5b950dca' -ex 'p *(uint32_t*)0xb459ec = 0x6d401548' -ex 'p *(uint32_t*)0xb459f0 = 0x6b18d9bf' -ex 'p *(uint32_t*)0xb459f4 = 0x7ad08b25' -ex 'p *(uint32_t*)0xb459f8 = 0x15d14f51' -ex 'p *(uint32_t*)0xb459fc = 0x8f86587a' -ex 'p *(uint32_t*)0xb45a00 = 0x774c4628' -ex 'p *(uint32_t*)0xb45a04 = 0xa77b0fcc' -ex 'p *(uint32_t*)0xb45a08 = 0x9d24ca05' -ex 'p *(uint32_t*)0xb45a0c = 0xf0995dcb' -ex 'p *(uint32_t*)0xb45a10 = 0xbf9f96f1' -ex 'p *(uint32_t*)0xb45a14 = 0xf320a719' -ex 'p *(uint32_t*)0xb45a18 = 0x3ad475dd' -ex 'p *(uint32_t*)0xb45a1c = 0xc6fac141' -ex 'p *(uint32_t*)0xb45a20 = 0x9' -batch

[clementgallet]

It sounds like I don't have access to your gitlab repo files, even being invited as Guest (or I failed to find them, which is totally possible).

[qixils]

https://gitlab.com/ben_lubar/tobytas ?

also, doesn't gitlab email you when you're invited to a repository?

[clementgallet]

I got the invitation, accepted it. Now I have access to the repo, but not the files ? This is all I can access: https://i.imgur.com/hITBNVB.png

[BenLubar]

Ah, weird, I thought Guest would give read-only access on a code sharing website, but I guess I was wrong.

You should have access to the files now.

[BenLubar]

At the current point in the TAS, the desync caused by fast forwarding will cause a game over followed by a "don't continue", so if that affects your ability to debug, the desync starts at 8:55 and is clearly visible in the fight at 11:11. The fight ends at 11:26 in the non-desynced version.

[BenLubar]

This can be considered lower priority now, because I've finished the Deltarune portion of the TAS and Undertale does not appear to exhibit this behavior.

I'm replacing Deltarune with this program that just waits a certain number of frames and exits:

https://gist.github.com/BenLubar/08acceb7f319f9a104a38de4d8edc849

[BenLubar]

This also causes the second half of Undertale (the song) to not play in New Home.

As far as I can tell, the underlying cause of this issue is how sleeps are handled when fast forwarding:

https://github.com/clementgallet/libTAS/blob/c7e2b66f0dae50a4d2600eeb519ab837dfa6aa75/src/library/sleepwrappers.cpp#L79-L81

If a secondary thread sleeps at all in fast forward mode and that thread's actions affect gameplay in any way, you're going to get a desync because the main thread doesn't ever sleep.