M1 Sunshine is not Responding To Keyboard and Mouse Events from Android and PC Moonlight Apps

[vrxgaminginc]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your issue described in the documentation?

• [X] I have read the documentation

Is your issue present in the nightly release?

• [X] This issue is present in the nightly release

Describe the Bug

1. Install Sunshine on M1 Mac using homebrew.
2. Connect to M1 sunshine with PC version of moonlight
3. Type or Move Mouse

Expected Behavior

Mouse pointer moves and keyboard works

Additional Context

No response

Host Operating System

macOS

Operating System Version

14.3.1

Architecture

other, n/a

Sunshine commit or version

0.23.0

Package

other (not listed)

GPU Type

n/a

GPU Model

Apple Silicon

GPU Driver/Mesa Version

Apple Silicon

Capture Method (Linux Only)

No response

Config

default/blank

Apps

No response

Relevant log output

no relevant logs, but this is the session log:
[2024:04:16:16:40:49]: Info: CLIENT CONNECTED
[2024:04:16:16:40:49]: Info: SDR color coding [Rec. 601]
[2024:04:16:16:40:49]: Info: Color depth: 8-bit
[2024:04:16:16:40:49]: Info: Color range: [MPEG]
[2024:04:16:16:40:49]: Error: opening microphone '' failed. Please set a valid input source in the Sunshine config.
[2024:04:16:16:40:49]: Error: Available inputs:
[2024:04:16:16:40:49]: Error: Unable to initialize audio capture. The stream will not have audio.
[2024:04:16:16:40:49]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:40:50]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:40:50]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:40:50]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:40:51]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:42:16]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:42:16]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:42:17]: Error: Encoder did not produce IDR frame when requested!
[2024:04:16:16:42:23]: Info: CLIENT DISCONNECTED

[zanjie1999]

not working for h264, but work fine with hevc h264 has this bug:

Error: Encoder did not produce IDR frame when requested!

so video is no update, and no events

[Cememer]

Same problem, but the buttons work. In my case, changing the encoder (Software / videotoolbox) and codec (H.264 / hevc) did not change anything.

[Hazer]

@zanjie1999 @Cememer can you try with the latest nightly and report back? https://github.com/LizardByte/Sunshine/pull/2485 was merged and it offers fix for streaming issues, h264 should work now, and in the case your mouse seems to not be moving, but it's actually the stream that's frozen, it may help too.

One more thing, have you included Sunshine in [System Settings > Privacy & Security > Privacy > Accessibility]? Or your terminal, if your running it from the terminal? There's a PR to improve that and make the request happen automatically, but it still has work to be done https://github.com/LizardByte/Sunshine/pull/2508, meanwhile you can add it manually.

The IDR frame error can be ignored regarding this issue. This is another unrelated issue being analysed.

[vrxgaminginc]

I updated to the latest nightly and installed via the Portfile. It prompted to add Terminal to accessibility. That may have been the issue. Thanks!

[zanjie1999]

@zanjie1999 @Cememer can you try with the latest nightly and report back? #2485 was merged and it offers fix for streaming issues, h264 should work now, and in the case your mouse seems to not be moving, but it's actually the stream that's frozen, it may help too.

One more thing, have you included Sunshine in [System Settings > Privacy & Security > Privacy > Accessibility]? Or your terminal, if your running it from the terminal? There's a PR to improve that and make the request happen automatically, but it still has work to be done #2508, meanwhile you can add it manually.

The IDR frame error can be ignored regarding this issue. This is another unrelated issue being analysed.

I don't know how to install nightly version with brew.

Now use 0.23.1, on macOS 12.7.1 on M1 Pro, give Screen Recording and Accessibility permissions to terminal, when use hevc encoder, all work fine , when use h264 encoder, video will only be updated once, and mouse not moving.

[Hazer]

@zanjie1999 so you have the same issue then, next release will fix it for you.

If you want to try it using brew, first uninstall your version, then you can download this file: https://github.com/LizardByte/Sunshine/actions/runs/9117980005/artifacts/1510816879

unzip the downloaded file, then in the same folder you unzipped, run to install: brew install --build-from-source sunshine.rb

edit: I did not test it, but I think head works too, just run: brew install sunshine --HEAD

[zanjie1999]

@zanjie1999 so you have the same issue then, next release will fix it for you.

If you want to try it using brew, first uninstall your version, then you can download this file: https://github.com/LizardByte/Sunshine/actions/runs/9117980005/artifacts/1510816879

unzip the downloaded file, then in the same folder you unzipped, run to install: brew install --build-from-source sunshine.rb

edit: I did not test it, but I think head works too, just run: brew install sunshine --HEAD

no tag with name "nightly", so I make a fork and install

sunshine.rb :

require "language/node"

class Sunshine < Formula
  desc "Self-hosted game stream host for Moonlight"
  homepage "https://app.lizardbyte.dev/Sunshine"
  url "https://github.com/zanjie1999/Sunshine.git",
    tag: "20240517"
  version "0.23.1.e6e4dcf"
  license all_of: ["GPL-3.0-only"]
  head "https://github.com/zanjie1999/Sunshine.git", branch: "nightly"

  depends_on "boost" => :build
  depends_on "cmake" => :build
  depends_on "node" => :build
  depends_on "pkg-config" => :build
  depends_on "curl"
  depends_on "miniupnpc"
  depends_on "openssl"
  depends_on "opus"

  def install
    ENV["BRANCH"] = "nightly"
    ENV["BUILD_VERSION"] = ""
    ENV["COMMIT"] = "e6e4dcf0170615f100987809258a7d3dd33b69a6"

    args = %W[
      -DBUILD_WERROR=ON
      -DCMAKE_INSTALL_PREFIX=#{prefix}
      -DOPENSSL_ROOT_DIR=#{Formula["openssl"].opt_prefix}
      -DSUNSHINE_ASSETS_DIR=sunshine/assets
      -DSUNSHINE_BUILD_HOMEBREW=ON
      -DTESTS_ENABLE_PYTHON_TESTS=OFF
    ]
    system "cmake", "-S", ".", "-B", "build", *std_cmake_args, *args

    cd "build" do
      system "make", "-j"
      system "make", "install"
      bin.install "tests/test_sunshine"
    end
  end

  service do
    run [opt_bin/"sunshine", "~/.config/sunshine/sunshine.conf"]
  end

  def caveats
    <<~EOS
      Thanks for installing Sunshine!

      To get started, review the documentation at:
        https://docs.lizardbyte.dev/projects/sunshine/en/latest/

      Sunshine can only access microphones on macOS due to system limitations.
      To stream system audio use "Soundflower" or "BlackHole".

      Gamepads are not currently supported on macOS.
    EOS
  end

  test do
    # test that the binary runs at all
    system "#{bin}/sunshine", "--version"

    # run the test suite
    # cannot build tests with python tests because homebrew destroys the source directory
    system "#{bin}/test_sunshine", "--gtest_color=yes"
  end
end

i test on 8Gen2 phone with hevc and h264，all work fine (video, touch, mouse) but I test on MT8183 ChromeBook with h264, no video out (black screen) or video will only be updated once and show "Poor connection to computer", but this MT8183 ChromeBook can work fine with Sunshine Windows version, I don't know where the problem lies either.

log:

$ sunshine

[sw_preset] -- [fast]
[vt_software] -- [disabled]
[audio_sink] -- [Loopback Audio]
[sw_tune] -- [fastdecode]
[locale] -- [zh]
[2024:05:17:16:51:39]: Info: Sunshine version: 0.23.1.e6e4dcf0170615f100987809258a7d3dd33b69a6
[2024:05:17:16:51:39]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:05:17:16:51:39]: Info: Trying encoder [videotoolbox]
[2024:05:17:16:51:39]: Info: Detecting displays
[2024:05:17:16:51:39]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2024:05:17:16:51:39]: Info: Detected display: RTK QHD (id: 2) connected: true
[2024:05:17:16:51:39]: Info: Configuring selected display (1) to stream
[2024:05:17:16:51:39]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:39]: Info: Color depth: 8-bit
[2024:05:17:16:51:39]: Info: Color range: [JPEG]
[2024:05:17:16:51:39]: Warning: [h264_videotoolbox @ 0x146707240] PrioritizeEncodingSpeedOverQuality property is not supported on this device. Ignoring.
[2024:05:17:16:51:39]: Info: [h264_videotoolbox @ 0x146707240] This device does not support the AllowOpenGop option. Value ignored.
[2024:05:17:16:51:39]: Info: [h264_videotoolbox @ 0x146707240] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:40]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:40]: Info: Color depth: 8-bit
[2024:05:17:16:51:40]: Info: Color range: [JPEG]
[2024:05:17:16:51:40]: Info: [hevc_videotoolbox @ 0x116608640] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:40]: Info: [hevc_videotoolbox @ 0x116608640] This device does not support the qmax option. Value ignored.
[2024:05:17:16:51:41]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:41]: Info: Color depth: 8-bit
[2024:05:17:16:51:41]: Info: Color range: [JPEG]
[2024:05:17:16:51:41]: Error: Couldn't open [av1_videotoolbox]
[2024:05:17:16:51:41]: Info: Detecting displays
[2024:05:17:16:51:41]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2024:05:17:16:51:41]: Info: Detected display: RTK QHD (id: 2) connected: true
[2024:05:17:16:51:41]: Info: Configuring selected display (1) to stream
[2024:05:17:16:51:41]: Info: SDR color coding [Rec. 709]
[2024:05:17:16:51:41]: Info: Color depth: 10-bit
[2024:05:17:16:51:41]: Info: Color range: [JPEG]
[2024:05:17:16:51:41]: Info: [hevc_videotoolbox @ 0x116608710] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:41]: Info: [hevc_videotoolbox @ 0x116608710] This device does not support the qmax option. Value ignored.
[2024:05:17:16:51:41]: Info:
[2024:05:17:16:51:41]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:05:17:16:51:41]: Info:
[2024:05:17:16:51:41]: Info: Found H.264 encoder: h264_videotoolbox [videotoolbox]
[2024:05:17:16:51:41]: Info: Found HEVC encoder: hevc_videotoolbox [videotoolbox]
[2024:05:17:16:51:41]: Error: Couldn't find any of the following libraries: [libavahi-common.3.dylib, libavahi-common.dylib]
[2024:05:17:16:51:41]: Info: Configuration UI available at [https://localhost:47990]
[2024:05:17:16:51:42]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:05:17:16:51:42]: Info: Trying encoder [videotoolbox]
[2024:05:17:16:51:42]: Info: Detecting displays
[2024:05:17:16:51:42]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2024:05:17:16:51:42]: Info: Detected display: RTK QHD (id: 2) connected: true
[2024:05:17:16:51:42]: Info: Configuring selected display (1) to stream
[2024:05:17:16:51:42]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:42]: Info: Color depth: 8-bit
[2024:05:17:16:51:42]: Info: Color range: [JPEG]
[2024:05:17:16:51:42]: Warning: [h264_videotoolbox @ 0x126605410] PrioritizeEncodingSpeedOverQuality property is not supported on this device. Ignoring.
[2024:05:17:16:51:42]: Info: [h264_videotoolbox @ 0x126605410] This device does not support the AllowOpenGop option. Value ignored.
[2024:05:17:16:51:42]: Info: [h264_videotoolbox @ 0x126605410] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:43]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:43]: Info: Color depth: 8-bit
[2024:05:17:16:51:43]: Info: Color range: [JPEG]
[2024:05:17:16:51:43]: Info: [hevc_videotoolbox @ 0x14670f320] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:43]: Info: [hevc_videotoolbox @ 0x14670f320] This device does not support the qmax option. Value ignored.
[2024:05:17:16:51:43]: Info: SDR color coding [Rec. 601]
[2024:05:17:16:51:43]: Info: Color depth: 8-bit
[2024:05:17:16:51:43]: Info: Color range: [JPEG]
[2024:05:17:16:51:43]: Error: Couldn't open [av1_videotoolbox]
[2024:05:17:16:51:43]: Info: Detecting displays
[2024:05:17:16:51:43]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2024:05:17:16:51:43]: Info: Detected display: RTK QHD (id: 2) connected: true
[2024:05:17:16:51:43]: Info: Configuring selected display (1) to stream
[2024:05:17:16:51:43]: Info: SDR color coding [Rec. 709]
[2024:05:17:16:51:43]: Info: Color depth: 10-bit
[2024:05:17:16:51:43]: Info: Color range: [JPEG]
[2024:05:17:16:51:43]: Info: [hevc_videotoolbox @ 0x14670f880] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:43]: Info: [hevc_videotoolbox @ 0x14670f880] This device does not support the qmax option. Value ignored.
[2024:05:17:16:51:43]: Info:
[2024:05:17:16:51:43]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:05:17:16:51:43]: Info:
[2024:05:17:16:51:43]: Info: Found H.264 encoder: h264_videotoolbox [videotoolbox]
[2024:05:17:16:51:43]: Info: Found HEVC encoder: hevc_videotoolbox [videotoolbox]
[2024:05:17:16:51:43]: Info: Executing [Desktop]
[2024:05:17:16:51:43]: Info: CLIENT CONNECTED
[2024:05:17:16:51:43]: Info: Detecting displays
[2024:05:17:16:51:43]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2024:05:17:16:51:43]: Info: Detected display: RTK QHD (id: 2) connected: true
[2024:05:17:16:51:43]: Info: Configuring selected display (1) to stream
[2024:05:17:16:51:43]: Info: SDR color coding [Rec. 709]
[2024:05:17:16:51:43]: Info: Color depth: 8-bit
[2024:05:17:16:51:43]: Info: Color range: [MPEG]
[2024:05:17:16:51:43]: Warning: [h264_videotoolbox @ 0x1466c40e0] PrioritizeEncodingSpeedOverQuality property is not supported on this device. Ignoring.
[2024:05:17:16:51:43]: Info: [h264_videotoolbox @ 0x1466c40e0] This device does not support the AllowOpenGop option. Value ignored.
[2024:05:17:16:51:43]: Info: [h264_videotoolbox @ 0x1466c40e0] This device does not support the qmin option. Value ignored.
[2024:05:17:16:51:44]: Warning: audio_control_t::set_sink() unimplemented: Loopback Audio
[2024:05:17:16:51:44]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY
[2024:05:17:16:51:44]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:44]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:44]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:45]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:45]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:45]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:46]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:46]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:46]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
[2024:05:17:16:51:47]: Error: Encoder did not produce IDR frame when requested!
^C[2024:05:17:16:51:56]: Info: Interrupt handler called
[2024:05:17:16:51:56]: Warning: audio_control_t::set_sink() unimplemented: Loopback Audio

[Cememer]

@zanjie1999 @Cememer can you try with the latest nightly and report back? #2485 was merged and it offers fix for streaming issues, h264 should work now, and in the case your mouse seems to not be moving, but it's actually the stream that's frozen, it may help too.

One more thing, have you included Sunshine in [System Settings > Privacy & Security > Privacy > Accessibility]? Or your terminal, if your running it from the terminal? There's a PR to improve that and make the request happen automatically, but it still has work to be done #2508, meanwhile you can add it manually.

The IDR frame error can be ignored regarding this issue. This is another unrelated issue being analysed.

I confirm that the mouse issue has been fixed in the latest Nightly version! Thanks for the commit!

[Hazer]

@zanjie1999 if you need support with the Chromebook, please open another issue please. The main issue here is closed.