Mouse button events are inconsistent or don't work on 1.6 Alpha

[eygraber]

Describe the bug Using the 4 different methods of handling pointer events for mouse buttons results in either broken behavior, or no behavior at all.

Tested with JVM, and wasmJs/js on Chrome (Ubuntu 22.04).

The code for testing all of these is:

var t by remember { mutableStateOf("Hello, world!") }

Scaffold { contentPadding ->
  Box(
    modifier = Modifier
         .fillMaxSize()
         .padding(contentPadding),
    contentAlignment = Alignment.Center,
  ) {
    Text(text = t)
  }
}

The following modifiers were tested by adding them after the padding modifier.

onPointerEvent(
  eventType = PointerEventType.Press,
  pass = PointerEventPass.Initial,
  onEvent = { event ->
    t = event.button?.index?.toString() ?: "null"
  }
)

JVM works, but only with PointerEventType.Press (and uses different values on Linux vs Windows/Mac)

wasmJs/js prints "null" for everything

pointerInput(Unit) {
  awaitEachGesture {
    t = currentEvent.button?.index?.toString() ?: "null"
  }
}

All targets freeze when running this and need to be killed by the process manager

pointerInput(Unit) {
  awaitPointerEventScope {
    while(true) {
      val event = awaitPointerEvent()
        t = event.button?.index?.toString() ?: "null"
      }
  }
}

JVM works

wasmJs/js prints "null" for everything

onClick(matcher = PointerMatcher.mouse(PointerButton(0)), onClick = { t = "Mouse Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(1)), onClick = { t = "Mouse Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(2)), onClick = { t = "Mouse Middle" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(7)), onClick = { t = "Mouse Thumb" })

JVM works (although mouse back and front and thumb aren't fired unless the left, right, or middle mouse button are being held down) wasmJs/js t never changes

[eymar]

Thank you for reporting this!

[eygraber]

@eymar I saw in https://github.com/JetBrains/compose-multiplatform-core/pull/972#issuecomment-1929187494 that there is refactoring work for input events for ios and web. Will this issue be addressed for wasm or JVM as part of that, or is it something separate?

[eymar]

@eymar I saw in JetBrains/compose-multiplatform-core#972 (comment) that there is refactoring work for input events for ios and web. Will this issue be addressed for wasm or JVM as part of that, or is it something separate?

This issue is rather separate. The refactoring won't bring new features/fixes itself, but it should make them easier to implement

[eymar]

@eygraber , apparently 1.6.10-dev1583 has it already fixed. I guess the refactoring indeed brought the fix too.

UPD: fixed in 1.6.10-dev1578 and newer

[eygraber]

Thanks! Still a couple of things not working on 1.6.10-dev1584:

onPointerEvent(
  eventType = PointerEventType.Press,
  pass = PointerEventPass.Initial,
  onEvent = { event ->
    t = event.button?.index?.toString() ?: "null"
  }
)

1. On WASM in Chrome, the horizontal scroll and mouse thumb button isn't detected. Mouse forward and back are 4 and 3 respectively
2. On JVM mouse forward and back are 6 and 5 respectively

pointerInput(Unit) {
              awaitEachGesture {
                t = currentEvent.button?.index?.toString() ?: "null"
              }
}

1. On WASM in Chrome the browser tab is frozen (probably for the same reason as #2)
2. On JVM null is printed in an infinite loop

pointerInput(Unit) {
  awaitPointerEventScope {
    while(true) {
      val event = awaitPointerEvent()
        t = event.button?.index?.toString() ?: "null"
      }
  }
}

1. On WASM in Chrome mouse move, mouse left click, and vertical and horizontal scrolling is a 0
2. On JVM mouse forward and back are 6 and 5 respectively; mouse move and vertical scrolling is null

onClick(matcher = PointerMatcher.mouse(PointerButton(0)), onClick = { t = "Mouse Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(1)), onClick = { t = "Mouse Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(2)), onClick = { t = "Mouse Middle" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(7)), onClick = { t = "Mouse Thumb" })

1. On WASM in Chrome the horizontal scroll and mouse thumb button isn't detected, and mouse forward and back are detected as horizontal scroll
2. On JVM mouse forward, back and thumb are only detected if the left, middle, or right mouse button is being held down

[eymar]

On WASM in Chrome, the horizontal scroll and mouse thumb button isn't detected. Mouse forward and back are 4 and 3 respectively

"horizontal scroll" - do you mean holding Shift and moving the mouse wheel? Or is there a special mouse button for horizontal scroll?

---

pointerInput(Unit) {
   awaitEachGesture {
       t = currentEvent.button?.index?.toString() ?: "null"
    }
}

here, it freezes on jvm for me as well. the block passed into awaitEachGesture never suspends. Adding any await call makes it work correctly on jvm: awaitPointerEvent() (something else to be fixed for web though).

---

[eymar]

I'm not sure about this snippet:

.onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })

PointerButton 3 and 4 are Back and Forward respectively:

val Back = PointerButton(3)
val Forward = PointerButton(4)

But you expect back and front clicks on 5 and 6.

https://github.com/JetBrains/compose-multiplatform-core/blob/4b5650e600ea0612a899478317a5ba7fa775ad13/compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/input/pointer/PointerButton.skiko.kt#L29

The pointer button value isn't always the same on all platforms. For example for desktop, there's such mapping:

https://github.com/JetBrains/compose-multiplatform-core/blob/085c24ab598923cdb859f5a2a997d5464420c56a/compose/ui/ui/src/desktopMain/kotlin/androidx/compose/ui/scene/ComposeSceneMediator.desktop.kt#L718

---

Does it clarify the PointerButton usage a bit?

[eygraber]

"horizontal scroll" - do you mean holding Shift and moving the mouse wheel? Or is there a special mouse button for horizontal scroll?

My mouse has a dedicated scroll wheel for horizontal scrolling.

PointerButton 3 and 4 are Back and Forward respectively

Not always 😅

On Linux the codes are different, which I believe is an issue from Swing / awt.

There's an issue for that where I linked a spreadsheet with some investigations I did to see what codes are fired for which action across OS and mouse manufacturer.

[eymar]

I see. Thanks for sharing more details.

currentEvent.button?.index is not 1 to 1 mapping to the value from OS (awt). It's remapped for compose purposes here

Probably the internal remapping should involve the info about the host OS.

Also, now I see a problem that manually creating a PointerButton(index) for onClick purposes can be error-prone, because of the implicit remapping by Compose (desktop) under the hood. The goal of PointerButton(index) was to make its values common for all Compose platforms.

---

I think in the scope of this issue, I'll fix the web part:

• button will be null in all events except press and release (same as on Desktop). 0 value is indeed a bug.

A couple of questions:

1. Have you figured out the freeze problem?

pointerInput(Unit) {
  awaitEachGesture {
    // suspend call is missing that's why it freezes
    t = currentEvent.button?.index?.toString() ?: "null"
  }
}

2. I'll try to get a more advanced mouse. Is horizontal scroll button/wheel clickable like middle mouse wheel? Does scrolling with horizontal scroll weel work in Compose for you (at least, can you see the pointer event)? I'm wondering what mouse button index does the browser send for this button when you click it?

[eygraber]

1. I haven't figured out the freezing problem, but I didn't spend too much time on it.
2. On my mouse the horizontal scroll wheel isn't clickable (and it hasn't been on any mouse that I've tried). I don't remember if the horizontal scroll wheel actually scrolls. I'll try it out.

[eygraber]

In regards to:

On Linux the codes are different, which I believe is an issue from Swing / awt.

Are there any plans to address that in Compose?

[eygraber]

Does scrolling with horizontal scroll weel work in Compose for you (at least, can you see the pointer event)?

Using 1.6.10-beta02 and Modifier.horizontalScroll(rememberScrollState()) the horizontal scroll wheel works! On JVM I have to hold down shift and scroll the horizontal scroll wheel.

[eygraber]

@eymar checking in on https://github.com/JetBrains/compose-multiplatform/issues/4134#issuecomment-2080152772

In regards to:

On Linux the codes are different, which I believe is an issue from Swing / awt. Are there any plans to address that in Compose?

[eymar]

It seems the Linux issue is still in the backlog. We'll revise it during the next planning.

[okushnikov]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.