libTAS
libTAS copied to clipboard
clementgallet
FS-UAE desync
The release that is available to apt seems to be the latest stable, so simply installing it using sudo apt install fs-uae should be enough to verify this problem.
In Documents/Floppies I have Turrican (1990)(Rainbow Arts)[cr TRSi].adf:
CRC32: 4B863522
MD5: B4826447461DBEF37994D04B40814BF9
SHA-1: 2C6286CDB96D073BA87E5202FBDFFB5F78D81040
901,120 bytes
Kickstart ROM doesn't seem to be needed as an implementation of it is embedded into fs-uae.
Here's the config in Documents/Configurations: Default.zip
Here's the movie I'm trying to play (nothing in lib options or command line arguments, I tried feeding the config explicitly from command line, doesn't change anything): fs-uae.zip
Every time I replay it, I see different outcome. And not even once it matched what I got when recording it. I noticed it randomly stutters around frames 2-4 even if you start unpaused. If you start paused, the first frame it pauses at changes between those 3. It seemed to be a little but more consistent while dumping avi, but still never synced.
This is using the latest build of libTAS. I couldn't find a libTAS option that improves sync, but I didn't try all the fs-uae options either. There's too many of them, and I wouldn't know what I'm doing.
There is at least a problem at startup with the code that initializes multiple mice (manymouse 3rd party library). The mouse init and input polling is done in a separate thread, and the main thread waits with usleep() calls until all mice have been detected.
This code is unnecessary, because fs-uae also reads inputs from the system mouse. There doesn't seem to be a configuration option to disable it. Also, the link above is to version 2.6, but when looking at the master commit, the manymouse code does not seem to be executed anymore. It was removed in ~~2.6.9dev~~ 2.9.6dev version (current Debian testing version is 2.8.4).
I was testing on FS-UAE 2.8.4. Oh and I even have mouse input disabled in libTAS.
Sorry, I meant it was removed in 2.9.6dev, so after 2.8.4. Having mouse disabled does not matter unfortunately. It still creates the thread.
I tried with https://launchpad.net/~fengestad/+archive/ubuntu/devel/+files/fs-uae_2.9.7~dev4-0bionic_amd64.deb
It pauses at frame 1 now, but still never works the same.
After a bit a research, here's what I understand. The program has a main thread doing the rendering and an emulator thread that generates each frame.
If we have video_sync on, the main thread renders continuously, using the video_sync_method parameter (eventually sleep, using fence, etc.) but does not check if the rendered frame is different from the previous one. So when fast-forwarding, we are getting multiple duplicate frames (relevent code in render_iteration_vsync()).
If we have video_sync off, then the main thread waits until a new frame is available before displaying it (in fs_ml_render_iteration()). This is good, but there is an issue with the first emulated frame, as the main thread waits for the emulator thread, but the later waits for time to pass. The main thread uses glib API (here g_cond_wait_until()) instead of pthread API (pthread_cond_timedwait()). As a result, we don't catch the implied sleep, so we don't advance time and the program softlocks. Setting time tracking > main thread > clock_gettime() allow the program to continue (although the freeze is annoyingly long).
Once the first frame is done, both threads are synced regarding virtual time, so there is no problem anymore, and the emulator seems to be deterministic. The difference between libTAS frame count and emulator frame count is always 8 in my case, and the garbled blue screen appears at frame 1071 for me everytime.
We could implement part of glib API to help this, but I'm not sure what to put in g_cond_wait_until(). If we want to prevent softlocks while being as deterministic as possible, we could check the conditional variable for a "long" time, then if it fails, advance time for the whole duration and either returns or do another check on the conditional variable.
Still never syncs to me. I removed the option setting video_sync, now it pauses on frame 7 for a minute, but otherwise it's still different every time. I even recorded a new movie, and result is still that it never plays the same way. Setting just clock_gettime() or all the time options doesn't help.
I also noticed that it pauses at frame 1 even if I have Pause unchecked in libTAS. When I click on the empty Pause checkbox, it flashes for a moment and the emulator reaches frame 7. IIRC Mari0 behaves the same way. Dunno if it matters. I think the older version of fs-uae didn't have this.
I've added an option for handling wait timeouts in 54fae7a2ad5359e9dad4f4bb1792278750382d8d. You can set Runtime > Wait timeout to Finite waits so that you don't have the freeze at the beginning.
Using this setting, I'm getting consistent game loading. However, there's a different kind of desync now. Although the movements of my character are always the same, the type, number and behavior of enemies are completely different each time. It may have to do with the PRNG implemented in the emulator.
PRNG values are consistent every time. So I have no idea. It is possible that the emulator is intrinsically non-deterministic (even if single-threaded) and that libTAS does not fix it? I tried to use the input replay option of UAE by patching FS-UAE to forward the relevant parameters, but I failed.
Mhhh, using savestates, fast-forwarding during gameplay consistently affects the enemy patterns.
I'd expect at least something helpful to be printed in the messages that libTAS can output. Going from that I believe there's something that we should be able to either patch out or prove that it won't work unless totally reworked.
I managed to patch fs-uae to get deterministic behavior! The problem is with the video_sync option:
-
if the option is on, and additionally if the screen framerate matches the game framerate, the emulator will enable a full video sync, and the emulation thread will always wait for the rendering thread to render a frame before processing the next one. However, if the rendering thread wants to draw before the emulation thread finished its processing, the rendering thread will draw a duplicate frame, which will mess up the timing in libTAS (each draw increments the time).
-
if the option is off, the rendering thread will never draw duplicate frames. However, the emulation thread will not wait for the rendering thread to draw the frame, it will start processing the next frame, with incorrect system time values in libTAS.
So I copied the wait on conditional variable in fs_ml_frame_update_end() to the case when g_fs_ml_video_sync is off, and added the conditional variable signaling from the vsync case to the non-vsync case, and it worked fine! The two threads are synchronized.
What's your setup to make it rerecord? When I use one vsync option the game goes to the ending screen from the start, and when I use another, I load a state and fs-uae crashes.
Sorry I didn't get your message.
I set video_sync = 0 in Default.fs-uae file. Otherwise, I use standard libTAS config, except for Runtime > Wait timeout > Finite waits.
Here my modified libfsemu/src/ml/render.c : https://gist.github.com/clementgallet/b96fea3a3c8f52fc37856835fbc47720
Since I don't understand what that patch does internally, but I want to send it as a PR to fs-uae, can you come up with some explanation that would convince the upstream dev? Basically, why should they want this patch?
They may not want that exact patch, because it decreases the efficiency of having separate threads if each one is waiting for the other. It is not useful for casual play. What do you think about making this an option?
You mean an option in fs-uae? That should be okay for all parties, provided they even care enough to accept such an option.
There's actually a deterministic option that doesn't do much, so I reused it to enforce a sequential execution of the rendering and emulation threads (https://github.com/FrodeSolheim/fs-uae/pull/195)
I updated fs-uae and pushed https://github.com/TASEmulators/fs-uae/commits/v3.1.66_f5bf027/ but when running in libTAS it gives this strange warning:
[libTAS f:0] Thread 15890 (main) OpenGL vendor: Mesa
[libTAS f:0] Thread 15890 (main) OpenGL renderer: llvmpipe (LLVM 15.0.7, 256 bits)
WARNING: No HW OpenGL driver: llvmpipe (LLVM 15.0.7, 256 bits)
and only runs at 10fps with this config Default.zip
Does any of it sound suspicions?
EDIT: Turns out the speed was so low because I was running A4000. Runs above normal fps with A500.
So I tried recording a movie in both original patch and an updated build, and unfortunately I'm still getting the same desyncs as before.
I recorded this movie with a bunch of savestates and saveloads, but after replaying it I get different item spawns and behavior of flying enemies every time I replay it, as well as after loading read-only savestates.
I used this fs-uae config and Finite waits.