Can you add support for mobile gyroscope？

[wswtst2007]

Can you add support for mobile gyroscope？

[sudhanshugairola]

Not support with host. Motion sensing not possible yet with rtsp.

[anthonywu1234]

Rewasd junior app can using the mobile gyroscope as ns pro/DS4 controller gyro by IP connecting.But moonlight-steam's virtual controller always occupy the first place in my PC lead to Rewasd junior app fail.Can you add a setting let the user choose not to open the moonlight-steam's virtual handle?

[baflo]

Seems like moonlight-qt supports gyroscope/motion now. Any chance this can be adopted on the Android version? https://github.com/moonlight-stream/moonlight-qt/issues/960#issuecomment-1628294952

[cgutman]

Gamepad motion is already supported with 5c6eaf2602479d0a379c55cbbf77ab2cff16050e, but I'm leaving this issue open until I implement the plan detailed here

[CnC-ode]

hello

@cgutman is there any nightly builds for the android version? if not, is there any build including this commit 5c6eaf2?

Edit1: Ok, couldn't find any info about nightly builds for the android repo, so I stopped being lazy and compiled a debug apk!

Couldn't get host to read the gyro data from my client thought... Tested with:

Client: latest main branch debug build (nvidia shield pro) Host: latest sunshine nightly build (0.20.0.f3a257b1509f0c39cd179319bae8fc8df92c5805) Controller: 8BitDo Pro 2 (connected through usb)

Tried emulating with ds4 on sunshine gamepads config and with forced moonlight gamepad driver checked and unchecked...

Note 1: I can confirm that the shield detects the gyro when the gamepad is connected through usb Note 2: I can confirm that when connecting controller on the host the gyro does function~~

Am I missing something?

Edit2: apparentely yes...:

// Hide gamepad motion sensor option when running on OSes before Android 12. // Support for motion, LED, battery, and other extensions were introduced in S. if (Build.VERSION.SDK_INT < Build.VERSION_CODES.S) { PreferenceCategory category = (PreferenceCategory) findPreference("category_gamepad_settings"); category.removePreference(findPreference("checkbox_gamepad_motion_sensors")); }

The code disables it for sdk versions < 31, and the shield is level 30...

@cgutman is there any fallback method that can be implemented on sdk versions < 31?

thanks

[baflo]

I couldn't make it work on an Android 12 phone (Pixel 3) neither with nonRootDebug nor nonRootRelease build target.

However, I managed to make it work using moonlight-qt on another PC with an Nintendo Switch Pro Controller. So, at least my host-side with Sunshine works.

Are the any other requirement except from having Android 12?

EDIT 1: Apparently, according to my debugging, the Switch Pro Controller's acceleration sensor is not detected by the SensorManager.

EDIT 2: 8bitdo sf30pro doesn't work either.

[bassderek]

I wasn't able to get this working either, installed sunshine nightly as well as built the up to date android app. Doing some debugging it seems to not even be detecting that the android device has a gyro sensor (this is on an abxylute handheld streaming device).

[baflo]

Exactly, while debugging the provided InputDevice's SensorManager doesnt return any sensor at all.

I wasn't able to verify thats Android supports gyro sensors of gamepads at all. Neither I could find definite documentation on this nor verify it with other games using my Switch Pro Controller and 8bitdo SF30 Pro.

[comanel]

@baflo & @cgutman i was able to make it work after tinkering a bit with it, inspired by https://github.com/Williangalvani/moonlight-android/tree/gyro_to_mouse_cleaned.

SensorManager won't return a sensor if you don't register a sensor event listener from the device itself. Once you do that, all you need is to override the onSensorChanged, get the (short)event.values.

        public void onSensorChanged(SensorEvent event) {
        if (event.sensor.getType() == Sensor.TYPE_GYROSCOPE) {
            float dx = event.values[0];
            float dy = event.values[1];
            float dz = event.values[2];

            short x = (short) (dx * -gyroSensitivity);
            short y = (short) (dy * gyroSensitivity);
            short z = (short) (dz * gyroSensitivity);
            controllerHandler.onGyro(x, y, z);

        }
    }

Afterwards, pass them on to controllerHandler.onGyro(), where instead of sending controllerInputPacket, i simply send controllerMotionEvent.

public void onGyro(short x, short y, short z) {
        //defaultContext.rightStickX = (short) Math.max(Math.min(x*1000, 32000), -32000);
        //defaultContext.rightStickY = (short) Math.max(Math.min(-y*1000, 32000), -32000);

        //sendControllerInputPacket(defaultContext);

//this is where we get fucky with pitch yaw roll, after trying multiple combinations this is what seems to do the trick
        conn.sendControllerMotionEvent((byte) defaultContext.controllerNumber,
                MoonBridge.LI_MOTION_TYPE_GYRO,
                -y,
                -x,
                z);
    }

I didn't fork anything, i just rebuilt the apk locally, because as @cgutman said, it's just a starting point. At the moment it doesn't have a fallback if the device doesn't have motion sensors, nor does it take into account device orientation, not to mention that the sensor listener will impact battery life since it's permanently registered... For me it works wonderfully, i use it with an OnePlus7 strapped into a razer Kishi and gyro stuff is mapped to whatever using Steam Input.

[comanel]

Oh, and any nightly greater than https://github.com/moonlight-stream/moonlight-android/commit/5c6eaf2602479d0a379c55cbbf77ab2cff16050e , tested with a DS4 controller, and gyro data is passed through flawlessly (as long as sunshine host emulates a ds4 controller)

[curtisfletcher]

@baflo & @cgutman i was able to make it work after tinkering a bit with it, inspired by https://github.com/Williangalvani/moonlight-android/tree/gyro_to_mouse_cleaned.

SensorManager won't return a sensor if you don't register a sensor event listener from the device itself. Once you do that, all you need is to override the onSensorChanged, get the (short)event.values.

        public void onSensorChanged(SensorEvent event) {
        if (event.sensor.getType() == Sensor.TYPE_GYROSCOPE) {
            float dx = event.values[0];
            float dy = event.values[1];
            float dz = event.values[2];

            short x = (short) (dx * -gyroSensitivity);
            short y = (short) (dy * gyroSensitivity);
            short z = (short) (dz * gyroSensitivity);
            controllerHandler.onGyro(x, y, z);

        }
    }

Afterwards, pass them on to controllerHandler.onGyro(), where instead of sending controllerInputPacket, i simply send controllerMotionEvent.

public void onGyro(short x, short y, short z) {
        //defaultContext.rightStickX = (short) Math.max(Math.min(x*1000, 32000), -32000);
        //defaultContext.rightStickY = (short) Math.max(Math.min(-y*1000, 32000), -32000);

        //sendControllerInputPacket(defaultContext);

//this is where we get fucky with pitch yaw roll, after trying multiple combinations this is what seems to do the trick
        conn.sendControllerMotionEvent((byte) defaultContext.controllerNumber,
                MoonBridge.LI_MOTION_TYPE_GYRO,
                -y,
                -x,
                z);
    }

I didn't fork anything, i just rebuilt the apk locally, because as @cgutman said, it's just a starting point. At the moment it doesn't have a fallback if the device doesn't have motion sensors, nor does it take into account device orientation, not to mention that the sensor listener will impact battery life since it's permanently registered... For me it works wonderfully, i use it with an OnePlus7 strapped into a razer Kishi and gyro stuff is mapped to whatever using Steam Input.

Would you please create a fork of what you created?

I tried to implemented what you said you did, but I couldn't get it to work. There weren't any errors, the gyro just still didn't work.

I've also got a razer kishi and it would be awesome if it had motion controls.

[mebalzer]

@curtisfletcher have you worked with @William Gavani’s joystick build? Does this work as is, or do I need to add your code to it as well?

Also, have tried to pass this on to ‘DS Windows’?

[comanel]

@curtisfletcher make sure sunshine emulates a DS4 controller. this is the branch you're looking for. https://github.com/comanel/moonlight-android/tree/newGyro

[curtisfletcher]

@curtisfletcher make sure sunshine emulates a DS4 controller

Thanks. I did try that but I think I just butchered the code. The functions you edited looked different in the version of moonlight I downloaded. It's okay, Im happy to use a ds4 for now until this is added as a feature.

[FishOrBear]

Can I use the motion control of the switch pro controller on a TV with Android 10.0?

I tried it but it doesn't seem to work

[curtisfletcher]

I couldn't get it to work either (but with a ds4)

I think it might require android 12. https://github.com/moonlight-stream/moonlight-android/releases/tag/v12.0

[nilsher]

Can't this one be closed ? V12.0 release says gyro has been implemented?

[baflo]

Can't this one be closed ? V12.0 release says gyro has been implemented?

I'm not sure. The release notes claim to have native support for PS4/5 controllers. At least for me it works neither with a Switch Pro Controller nor with an Android phone's integrated gyro.

Can anybody else confirm?

[nilsher]

I can confirm that Gyro works on an Ayn Odin 2 Android device. Had to enable "Emulate gamepad motion sensor"