ppy
Aspire beatmap video is not aligned correctly with gameplay
Describe the bug: This map is an aspire type of map that emphasizes the video in the background. But here, for some strange reason, the video is in a hurry for about 1.5-2 seconds. Screenshots or videos showing encountered issue: lazer gameplay: https://youtu.be/cJvUl7EoxAs
original: https://youtu.be/0ZXNxdxWZmo
map - https://osu.ppy.sh/beatmapsets/1503327#osu/3079907 osu!lazer version: 2021.623.0
@bdach I have information about another problematic aspire map, where there is a problem with music playback (assuming the map starts with negative time values), a storyboard problem, where storyboard objects are displayed much earlier than they should. I'm not sure if you know about this map now. Is it possible to create issue? This map - https://osu.ppy.sh/beatmapsets/1237363#osu/2571858
I will remove this comment after answering
Can you confirm this only occurs for you on this one beatmap, not others with video offsets?
The map used for the comparison above (https://osu.ppy.sh/beatmapsets/1376486#osu/2869013) has video offset 0, but I've taken it and manually edited one of the beatmap .osu files to set offset to -1 and there seems to be no discernible difference. So likely it's somehow specific to this one broken beatmap from the OP.
I experienced the same problem with this map in danser.
After digging into it, it turns out MP4/MOV container contains metadata called "edit list" which instructs parsers how to cut the video. Most players except QuickTime/VLC/FFPlay ignore it.
It looks like osu!stable ignores it as well, so videos with that data become misaligned in lazer. For now I'm unaware if are there other videos with similar problem.
ffmpeg CLI/ffplay have -ignore_editlist argument which fixes the problem but I don't know how to apply it with ffmpeg API.
Poking in ffmpeg source & docs it looks like the desired incantation should be something along the lines of av_opt_set_int(formatContext, "ignore_editlist", 1, 0) or similar? Have not tested, though.
Not sure how to jam that into framework, either... (do we just hardcode, or do we add some configuration primitive for it?)
IIRC almost everyone re-encodes the video with ffmpeg, even cutting without re-encode actually removes frames, so the only source of editlist would be if someone used QuickTime to trim the video (which seems to be the case here as the video still has audio attached).
The issue seems to affect not just loved aspire maps, but also a ranked mapset (potentially multiple). I don't know if the technical reason behind the issue is the same, but there seems to be the same outcome.
Affected mapset: https://osu.ppy.sh/beatmapsets/1313664#osu/2722655 The topdiff [Repetition] visualizes it best, it's especially noticeable when the girl in the video repeatedly jumps up and down following sliders (it happens around every chorus). Eyeballing it, the offset in lazer is maybe a couple tenths of a second, but it's a large enough difference to completely break the experience of cursordancing together with the video.
The map is playable just fine with 100% dim or without video, but it somewhat affects playability with video playing and dim set to lower values.
I haven't tested them extensively since they don't do the same "dance" gimmick, but these two mapsets seem to use the exact same background video file:
- https://osu.ppy.sh/beatmapsets/1222543
- https://osu.ppy.sh/beatmapsets/1336616
- I haven't checked mapsets for other gamemodes nor any unranked ones
I just wanted to bring this up since it's a ranked map that is affected.
I can't see the issue on beatmaps which @TechniKris mentioned locally. The beatmap in the OP looks clearly broken but not the purportedly "affected" mapset.
I can't see the issue on beatmaps which @TechniKris mentioned locally. The beatmap in the OP looks clearly broken but not the purportedly "affected" mapset.
Ah, I should've made a video showcasing the issue. Here, I made one now. I synced up gameplay, but the video on lazer is 8 frames (of this 60fps recording) ahead of stable
https://github.com/ppy/osu/assets/30842158/25a110eb-f6e2-4c49-aab8-f2814283ee52