Add more ways to seek to sample points

[OliBomby]

This feature unlocks a powerful UX that allows users to quickly edit multiple samples by double-clicking and using hotkeys, without having to move the mouse cursor.

[bdach]

I'm gonna be frank, I'm not sure I see what is "powerful" in this.

• For adding additional sounds, W~T keys are faster.
• For changing bank of all sounds at once, W~R keys are faster.
• For setting volume or setting separate normal/addition banks, clicking once is faster than clicking twice. And then if you want to change another sound, you have to move the mouse again anyway, albeit maybe less so than without the seek.

Is it the seek that's supposed to be making this "powerful"? Then let's maybe just have the popover seek every time?

In general I'm not sure this is going to be winning many favours. I've already heard from multiple feedback sources that the sample pieces on the timeline are difficult to work with when the timeline has high object density.

I'm honestly thinking we will inevitably have to go to the stable model of having clickable slider nodes themselves on playfield and/or timeline. That said I would want @peppy buy-in on this, because it has severe implications, such as killing the "all rulesets get a single unchanging implementation of timeline" premise, but I personally think that premise is dead in the water already if we wanna support everyone's use cases properly (case in point: scrolling rulesets and recurring complaints that you can't see scroll speed changes on timeline - which is because we moved them to timing screen).

[OliBomby]

I'm gonna be frank, I'm not sure I see what is "powerful" in this.

• For adding additional sounds, W~T keys are faster.
• For changing bank of all sounds at once, W~R keys are faster.

It's powerful in combination with these features. Mappers usually have a specific pattern of hitsounds in mind and need to apply different hitsounds to each object. The automatic seek allows them to: click - press keys for hs - click - press keys for hs - ... All without moving the mouse if its on the same snap. It optimizes the flow ever so slightly.

Is it the seek that's supposed to be making this "powerful"? Then let's maybe just have the popover seek every time?

I think this is also acceptable UX.

I'm honestly thinking we will inevitably have to go to the stable model of having clickable slider nodes themselves on playfield and/or timeline.

Does this imply removing the sample point pieces entirely? Would be good to know before I try redesigning them again to hopefully make them bearable to use.

I'm not against going to the stable model, it works pretty well and we can potentially have a separate hitsounding tab for more advanced hitsounds or modding.

it has severe implications, such as killing the "all rulesets get a single unchanging implementation of timeline" premise, but I personally think that premise is dead in the water already if we wanna support everyone's use cases properly (case in point: scrolling rulesets and recurring complaints that you can't see scroll speed changes on timeline - which is because we moved them to timing screen).

I'm also in favour of allowing ruleset-specific implementations of the timeline. I ran into issues with this back when https://github.com/ppy/osu/pull/21566

[bdach]

Does this imply removing the sample point pieces entirely? Would be good to know before I try redesigning them again to hopefully make them bearable to use.

I'm honestly not sure. I haven't really tried to come up with what the alternative would be yet. Maybe they would stay, but as display-only and not be directly interactable.

[peppy]

I don't know, this feels really awkward with its current implementation:

https://github.com/ppy/osu/assets/191335/1db68486-2143-4593-b3cb-75ffaf6e2614

Rather than this, would it not be better to have something like Alt+Left/Right skip to the next object (generally in editor) and making Shift+Alt+Left/Right bring up the samples edit box or similar?

[peppy]

As for a better path forward, I think there should be a way to expand the timeline out and reveal a hitsounding section (can take up as much height as required) or even a separate hitsounding mode as suggested by @OliBomby where using left/right arrows seeks between hitsound nodes (similar to keyframe skipping in a video editor).

[bdach]

I think there should be a way to expand the timeline out and reveal a hitsounding section (can take up as much height as required) or even a separate hitsounding mode

I would generally be on board with this (have floated the same previously) but I'm not super sure all users will be happy with just that. Some are used to being able to seamlessly hitsound while mapping and as such, these types of modality changes where you need to go into and out of a special hitsounding mode are not likely to work super well for them.

[OliBomby]

I've implemented both of your suggestions.

• It now seeks on click instead of double-click. I think this makes it a lot less jank.
• I've added alt + left/right to skip to the next object.
• I've added shift + alt + left/right to skip to the next sample point and automatically open it. This will be very useful for accessing those very densely packed sample points.

[peppy]

I'm not 100% sure about the auto scroll on single click.

[OliBomby]

I'm not 100% sure about the auto scroll on single click.

I'll ask some mappers for feedback about this. I'm also not 100% sure.

Next to your patch I also disabled Alt+Scroll to seek, which I had previously enabled. We probably don't want to disable the ability to change volume in the editor with Alt+Scroll.

Edit: I got feedback from DeviousPanda. He seems most excited about the ability to seek to nearby sample points with the keybinds. Particularly he liked the scroll functionality for seeking to nearby sample points, so I think I'll turn that into a configurable keybind so he can rebind it to the scrollwheel if he wants.

About the auto seek on single click. He said it brings lazer closer to stable functionality, so it might be a good change. I think its fine to leave it in then and see if people complain.

[peppy]

I don't know what's going on here but it looks pretty broken:

https://github.com/user-attachments/assets/1c7a9a74-2e29-4f9f-a1c3-c27c9c4d3b93

Probably needs investigation before considering adding this UX behaviour.

[OliBomby]

I don't know what's going on here but it looks pretty broken:

osu.2024-08-19.at.04.18.29.mp4 Probably needs investigation before considering adding this UX behaviour.

I fail to see what is broken in the video. It's supposed to seek to neighbouring hitobjects with Alt+left/right and seek to nearby sample control points with Alt+Shift+left/right. That seems to be happening in the video.

[Joehuu]

Adding an arrow to the popover makes the bug more apparent. With this diff:

diff --git a/osu.Game/Screens/Edit/Compose/Components/Timeline/SamplePointPiece.cs b/osu.Game/Screens/Edit/Compose/Components/Timeline/SamplePointPiece.cs
index 731fe8ae6a..81d5b2e0cb 100644
--- a/osu.Game/Screens/Edit/Compose/Components/Timeline/SamplePointPiece.cs
+++ b/osu.Game/Screens/Edit/Compose/Components/Timeline/SamplePointPiece.cs
@@ -12,6 +12,7 @@
 using osu.Framework.Graphics;
 using osu.Framework.Graphics.Containers;
 using osu.Framework.Graphics.Cursor;
+using osu.Framework.Graphics.Shapes;
 using osu.Framework.Graphics.UserInterface;
 using osu.Framework.Input.Events;
 using osu.Game.Audio;
@@ -145,6 +146,13 @@ public partial class SampleEditPopover : OsuPopover
             [Resolved(canBeNull: true)]
             private EditorBeatmap beatmap { get; set; } = null!;
 
+            protected override Drawable CreateArrow() => new Triangle
+            {
+                Size = new Vector2(20),
+                Anchor = Anchor.Centre,
+                Origin = Anchor.TopCentre,
+            };
+
             public SampleEditPopover(HitObject hitObject)
             {
                 this.hitObject = hitObject;

https://github.com/user-attachments/assets/a7f0114a-b678-4c3b-a2ad-fd0c1178d13e

Sometimes the popover doesn't go to the seeked object. Also I feel like adding the arrow is a good UX improvement because before I wouldn't know what I'm editing.

[OliBomby]

Ah I see the problem now. It seems I already fixed this in https://github.com/ppy/osu/pull/28772 but this PR wasn't updated since.

Sometimes the popover doesn't go to the seeked object. Also I feel like adding the arrow is a good UX improvement because before I wouldn't know what I'm editing.

I like that triangle. I'll add that as well.

[bdach]

The arrow key to move around thing seems pretty cool for whatever it's worth.

Adding the arrow to the popover though, not sure. Weird to be adding it only in one place. The reason why I didn't originally do this even though Popover supports it is that it doesn't work with shadows:

Note that the shadow draws over the arrow. It's not super trivial to fix.

Maybe we need selection state to show on the sample/velocity pieces themselves. Probably with something like a yellow border or whatever, to match blueprints. It's something that I had in mind before, in that it's kinda difficult to tell right now which pieces you're editing specifically at any given time, especially when multiple objects are selected at a time.

[OliBomby]

Adding the arrow to the popover though, not sure. Weird to be adding it only in one place. The reason why I didn't originally do this even though Popover supports it is that it doesn't work with shadows:

Note that the shadow draws over the arrow. It's not super trivial to fix.

Maybe we need selection state to show on the sample/velocity pieces themselves. Probably with something like a yellow border or whatever, to match blueprints. It's something that I had in mind before, in that it's kinda difficult to tell right now which pieces you're editing specifically at any given time, especially when multiple objects are selected at a time.

Seems like those solutions are all not trivial, so I'll keep it out of this PR.

I think the yellow border on the sample pieces makes sense if they are also individually selectable like hitobjects in the timeline.

[peppy]

I was also very tempted to add the speech bubble marker. I wouldn't mind if it was just applied here for now, but either or I guess.

[peppy]

@OliBomby I'm thinking we should change this to be Ctrl instead of Alt to match user expectations (ie. matches traversing text by words in a text box). What say you?

[OliBomby]

@OliBomby I'm thinking we should change this to be Ctrl instead of Alt to match user expectations (ie. matches traversing text by words in a text box). What say you?

That makes sense yeah. I'll go with Ctrl+arrow keys for hitobject seek and Alt+arrow keys for sample point seek.

The only pain is that if you want to change they hotkeys from arrow keys to scroll wheel there will not be an obvious conversion for the sample point seek since alt+scroll is already used for the volume control, so you'll have to go for the only remaining combination which is Alt+Shift+scroll. But I guess its not a big deal since you can also change the hotkeys for beat snap and distance spacing to free up hotkeys.

[peppy]

Can we not keep the ctrl-shift for sample points? I feel like we may want to keep alt for other usages in the future. Or leave it unused.

[OliBomby]

Can we not keep the ctrl-shift for sample points? I feel like we may want to keep alt for other usages in the future. Or leave it unused.

That also works I guess.