support
support copied to clipboard
pybricks
[Feature] Gamepad controller investigation
This is a spot for discussing gamepad connections, split off from https://github.com/pybricks/support/issues/262
> ### 4. BLE HID Peripherals
We may support connecting to generic HID devices over BLE (and possibly classic). Users could build on this to add specific mappings for certain devices like popular gaming consoles.
From https://github.com/pybricks/support/issues/191#issuecomment-1493256816 :
- From "Connecting to an Xbox one controller" (over BLE w/ ubluetooth) https://gist.github.com/CubeLegend/f77c6f0d8b2aaee78a30d2c6508cdaf2 :
import binascii
import ubluetooth
from micropython import const
_IRQ_CENTRAL_CONNECT = const(1)
_IRQ_CENTRAL_DISCONNECT = const(2)
_IRQ_GATTS_WRITE = const(3)
_IRQ_GATTS_READ_REQUEST = const(4)
_IRQ_SCAN_RESULT = const(5)
_IRQ_SCAN_DONE = const(6)
_IRQ_PERIPHERAL_CONNECT = const(7)
_IRQ_PERIPHERAL_DISCONNECT = const(8)
_IRQ_GATTC_SERVICE_RESULT = const(9)
_IRQ_GATTC_SERVICE_DONE = const(10)
_IRQ_GATTC_CHARACTERISTIC_RESULT = const(11)
_IRQ_GATTC_CHARACTERISTIC_DONE = const(12)
_IRQ_GATTC_DESCRIPTOR_RESULT = const(13)
_IRQ_GATTC_DESCRIPTOR_DONE = const(14)
_IRQ_GATTC_READ_RESULT = const(15)
_IRQ_GATTC_READ_DONE = const(16)
_IRQ_GATTC_WRITE_DONE = const(17)
_IRQ_GATTC_NOTIFY = const(18)
_IRQ_GATTC_INDICATE = const(19)
_IRQ_GATTS_INDICATE_DONE = const(20)
_IRQ_MTU_EXCHANGED = const(21)
_IRQ_L2CAP_ACCEPT = const(22)
_IRQ_L2CAP_CONNECT = const(23)
_IRQ_L2CAP_DISCONNECT = const(24)
_IRQ_L2CAP_RECV = const(25)
_IRQ_L2CAP_SEND_READY = const(26)
_IRQ_CONNECTION_UPDATE = const(27)
_IRQ_ENCRYPTION_UPDATE = const(28)
_IRQ_GET_SECRET = const(29)
_IRQ_SET_SECRET = const(30)
_ADV_IND = const(0x00)
_ADV_DIRECT_IND = const(0x01)
_ADV_SCAN_IND = const(0x02)
_ADV_NONCONN_IND = const(0x03)
_UART_SERVICE_UUID = ubluetooth.UUID("6E400001-B5A3-F393-E0A9-E50E24DCCA9E")
_UART_RX_CHAR_UUID = ubluetooth.UUID("6E400002-B5A3-F393-E0A9-E50E24DCCA9E")
_UART_TX_CHAR_UUID = ubluetooth.UUID("6E400003-B5A3-F393-E0A9-E50E24DCCA9E")
class BLESimpleCentral:
def __init__(self, ble):
self.gamepad_name = None
self.gamepad_addr_type = None
self.gamepad_addr = None
self._ble = ble
self._ble.active(True)
self._ble.irq(self.bt_irq)
def bt_irq(self, event, data):
if event == _IRQ_SCAN_RESULT:
# A single scan result.
addr_type, addr, adv_type, rssi, adv_data = data
if find_adv_name(adv_data) is None:
return
print('Address-Type: ', addr_type)
print('Address: ', [hex(n) for n in addr])
print('ADV-Type: ', adv_type)
print('RSSI in dBm: ', rssi)
print('ADV-Data: ', [hex(n) for n in adv_data])
print('ADV-Data (Name): ', find_adv_name(adv_data)) # Xbox Wireless Controller
print()
if find_adv_name(adv_data) == "Xbox Wireless Controller":
self.gamepad_name = find_adv_name(adv_data)
self.gamepad_addr_type = addr_type
self.gamepad_addr = addr
self._ble.gap_scan(None)
self.connect()
elif event == _IRQ_SCAN_DONE:
# Scan duration finished or manually stopped.
pass
elif event == _IRQ_PERIPHERAL_CONNECT:
# A successful gap_connect().
conn_handle, addr_type, addr = data
print("connect")
print(data)
print('Address-Type: ', addr_type)
print('Address: ', [hex(n) for n in addr])
print()
elif event == _IRQ_PERIPHERAL_DISCONNECT:
# Connected peripheral has disconnected.
conn_handle, addr_type, addr = data
print("disconnect")
print(data)
print('Address-Type: ', addr_type)
print('Address: ', [hex(n) for n in addr])
print()
# Find a device advertising the environmental sensor service.
def scan(self):
self._ble.gap_scan(5000, 100000, 25000, True)
def connect(self):
print(self.gamepad_addr)
self._ble.gap_connect(self.gamepad_addr_type, self.gamepad_addr)
def find_adv(adv_type, data):
i = 0
while i + 1 < len(data):
ad_structure_len = data[i]
ad_structure_type = data[i + 1]
ad_structure_payload = data[i + 2: i + ad_structure_len + 1]
if ad_structure_type == adv_type:
return ad_structure_payload
i += ad_structure_len + 1
return None
def find_adv_name(data):
n = find_adv(9, data)
if n:
return str(n, 'UTF-8') # Text
return None
def demo():
ble = ubluetooth.BLE()
central = BLESimpleCentral(ble)
central.scan()
def demo_connect():
ble = ubluetooth.BLE()
central = BLESimpleCentral(ble)
central.connect()
if __name__ == "__main__":
demo()
@all @mwinkler @cubelegend ( https://github.com/pybricks/support/issues/140#issuecomment-782938904)
- Does this work with an Xbox controller?
- Does this work with a Stadia controller?
- Does this work with a PS4 controller?
- Is there an / is this the HID Gamepad + Joystick(s) standard, or does there need to be configurable button remapping from device ids sort of like Steam PC?
- https://www.google.com/search?q=steam+controller+remap
- https://partner.steamgames.com/doc/features/steam_controller/getting_started_for_players
- https://www.google.com/search?q=steam+controller+remap
Originally posted by @westurner in https://github.com/pybricks/support/issues/262#issuecomment-1493260385
@westurner This seems to be especially for XBOX ONE controllers. I'm not 100% sure about the low level stuff, but as far as I know, XBOX controllers use XINPUT while most other controllers use DINPUT (DINPUT beeing HID based?). It won't work with the stadia controller as that one uses DINPUT and it won't work with a PS4 controller because those use BTC and not BLE.
I think that there is a button standard for XINPUT but not really for DINPUT. Of course, one could focus on the most popular gamepads. I don't think that there are many gamepads out there that use BLE.
Originally posted by @Tcm0 in https://github.com/pybricks/support/issues/262#issuecomment-1493262116
@westurner
Here is the output of the script (running on stock mindstorm firmware, ble xbox controller):
So connection seems to work
Originally posted by @mwinkler in https://github.com/pybricks/support/issues/262#issuecomment-1493281515
- [ ] https://github.com/pybricks/support/issues/995
- [ ] ENH: pybricks-code:
- Get the BLE Controller device ID
- Show which buttons are being activated (~ https://gamepad-tester.com/ https://www.google.com/search?q=gamepad-tester+github https://opengameart.org/content/free-keyboard-and-controllers-prompts-pack )
- (Remap buttons)
- Generate the Python code block and or ideally just a config dict (?) for the LEGO hub
- device ID
- HID ~profile
- button map
- https://en.wikipedia.org/wiki/PlayStation_4_technical_specifications#Input_and_output
- PS4: Bluetooth 2.1
- PS4 Slim: Bluetooth 4.0 (BLE)
- https://en.wikipedia.org/wiki/PlayStation_5#Hardware
- PS5: Bluetooth 5.1 (BLE)
- https://en.wikipedia.org/wiki/Xbox_Wireless_Controller#Third_revision_(2020)
- XboxOne Controller rev 1: No Bluetooth
- XboxOne Controller rev 2: Bluetooth 4.0
- XboxOne Controller rev 3 (2019-): Bluetooth (BLE)
The controller also supports Bluetooth Low Energy, and can be paired to a Bluetooth device and an Xbox device simultaneously.
- XboxOne Adaptive Controller: Bluetooth 4.1 (BLE) https://en.wikipedia.org/wiki/Xbox_Adaptive_Controller
- https://en.wikipedia.org/wiki/Google_Stadia#Controller
- Stadia: Bluetooth 5.0 (BLE) [2019-2023]
DirectInput (DINPUT), XInput (XINPUT), xlib
- https://en.wikipedia.org/wiki/DirectInput#DirectInput_vs_XInput
- https://www.pcgamingwiki.com/wiki/Glossary:Controller#Windows
XInput XInput is Microsoft's API for Xbox 360-compatible controllers such as the official Xbox 360 Controller.
- XInput wrapping:
- https://www.pcgamingwiki.com/wiki/Controller:Stadia_Controller#XInput_wrapping
- If necessary for a particular game, you can convert DirectInput to XInput with various apps including Steam Link on Linux (flatpak)
- xlib (X11; Linux) https://python-xlib.github.io/python-xlib_11.html#Selecting-Events
- https://github.com/r4dian/Xbox-Controller-for-Python/blob/master/xinput.py
- https://wiki.archlinux.org/title/Gamepad#Mimic_Xbox_360_controller
-
xboxdrv
-
- https://levelup.gitconnected.com/build-an-xbox-controller-abstraction-layer-in-python-using-xinput-api-aaf0f8d05ac2
- https://wiki.archlinux.org/title/Gamepad#Connect_Xbox_Wireless_Controller_with_Bluetooth
- https://github.com/atar-axis/xpadneo/#connection
- how to
bluetoothctl scan - https://github.com/atar-axis/xpadneo/blob/master/hid-xpadneo/src/hid-xpadneo.c
Force feedback support for XBOX ONE S and X gamepads via Bluetooth
- how to
- https://github.com/atar-axis/xpadneo/#connection
- https://wiki.archlinux.org/title/Gamepad#Device_permissions
- https://codeberg.org/fabiscafe/game-devices-udev
- https://pkgs.rpmfusion.org/cgit/nonfree/steam.git/tree/60-steam-input.rules (rpmfusion
steam-devices - https://github.com/ValveSoftware/steam-devices/blob/master/60-steam-input.rules
Originally posted by @westurner in https://github.com/pybricks/support/issues/262#issuecomment-1493283886
Does there need to be a *gamepad* Bluetooth HID in the pybricks firmware?
- https://github.com/adafruit/Adafruit_nRF52_Arduino/issues/228#issuecomment-463191651)
- https://github.com/sparkfun/circuitpython/blob/673d2fb4b7c5e077fc5579af1d249438d17d93ad/tools/hid_report_descriptors.py#L177-L207
- https://github.com/sparkfun/circuitpython/blob/673d2fb4b7c5e077fc5579af1d249438d17d93ad/tools/hid_report_descriptors.py#L246-L276
- https://learn.adafruit.com/custom-hid-devices-in-circuitpython?view=all#report-descriptors-3103536
- https://github.com/lemmingDev/ESP32-BLE-Gamepad
- https://github.com/DJm00n/ControllersInfo -- HID profiles for Xbox 360, Xbox One, PS4, PS4, Stadia, and Switch
- "HID over GATT Profile (HOGP) 1.0" https://www.bluetooth.org/docman/handlers/downloaddoc.ashx?doc_id=245141 :
1 Introduction The HID over GATT profile defines the procedures and features to be used by Bluetooth low energy HID Devices using GATT and Bluetooth HID Hosts using GATT. This profile is an adaptation of the USB HID specification [2] to operate over a Bluetooth low energy wireless link. This profile shall operate over an LE transport only. For BR/EDR, the Bluetooth Human Interface Device Profile Specification [9] shall be used. 1.1 Profile Dependencies This profile requires the Generic Attribute Profile (GATT), the Battery Service, the Device Information Service, and the Scan Parameters Profile. This specification can be used with Bluetooth Core Specification Version 4.0 [3] or later.
- "HID over GATT Profile (HOGP) 1.0" https://www.bluetooth.org/docman/handlers/downloaddoc.ashx?doc_id=245141 :
Originally posted by @westurner in https://github.com/pybricks/support/issues/262#issuecomment-1493287515
Nice to see this issue split off, and some investigation going on, still interested in this one.
@westurner, how far did you get with this? I remember concluding with the PyBricks guys (in a March maybe) that if some basic BT connectivity functionality would be cleaned up in the PyBricks code (along the lines of the code that connects to the lego remote), then we might be able to build on that to experiment with HID gamepad controllers.
In the above example ubluetooth is a micropython module, right? Is that available on PyBricks? Could it be? I remember us discussing with @dlech that the bluetooth module is not available and would be problematic, but maybe ubluetooth could be, so that we can build on it? What's the essential difference?
In MicroPython, ubluetooth is an alias for bluetooth so they are both names for the same module.
Oh, that's a pity, so nothing new there. Has there been any restructuring lately about BLE device connectivity code to make it more generally applicable for other purposes than connecting the lego remote?
All is not lost though, we're always working on Bluetooth --- @dlech did make significant progress on Bluetooth, just in a different way :wink:
@laurensvalk thanks for the update! It seems to me that the whole ble member is recently added in v3.3, right? Is that something that could gain more functionality in the future, such as some generic scan / connect methods to connect to peripheral devices?
Yes, it's definitely a step in the right direction. Indeed, the scanning procedure has been prepared for more generic use in the future.
If someone can sort this out and support both Xbox series, and PlayStation 5 controllers at minimum. I would make a donation to both pybricks and the individual developer of $500.00 each. Must be completed no later then end of Q1 2024.
@mozts2005 that's a generous offer, thanks! @laurensvalk, any chance of giving this some priority? As I said earlier, if the base BLE connectivity code would be there in some generic reusable form, I'd be happy to help with sorting out higher level HID processing part and @westurner might also be interested in helping in. And I'd not be doing it for the money, would even be okay with donating my part (if any) to PyBricks developers.
I don't know if you guys follow this Eurobricks thread about improving Powered Up stuff, but one of the top things that many people complain about is the lack of proportional physical remote, or any kind of 3rd party alternative that can be used without a smart device, so I'd argue that there is more interest than visible from these discussions here on Github. https://www.eurobricks.com/forum/index.php?/forums/topic/195601-lets-fix-powered-up/
Thanks @mozts2005! Let's see if we can make this happen. Would you mind dropping a note at [email protected]? I'd like to confirm a few details to be sure. Thanks!
Thanks also @gyenesvi for your kind follow-up. We are indeed aware of community comments about this. To drop a bit of a hint, we are actively working on a huge update behind the scenes. It will address another big request from the community (perhaps even more popular than Bluetooth). :smile:
That's promising and interesting to hear @laurensvalk, thanks, and keep us posted about the developments!
What's the resolution of this? I see it marked completed, but with what action? Is the investigation completed, or is the feature actually implemented?
Sorry, that must have happened accidentally as I was moving other linked issues around. Rest assured, this is still on our list :smile:
If someone can sort this out and support both Xbox series, and PlayStation 5 controllers at minimum. I would make a donation to both pybricks and the individual developer of $500.00 each. Must be completed no later then end of Q1 2024.
Originally posted by @mozts2005 in https://github.com/pybricks/support/issues/1024#issuecomment-1712963131
I hope the current version meets your needs partially! No matter what you decide, I hope you enjoy the new update :slightly_smiling_face:
All Xbox series controllers since 2016 are supported. With Technic Hub, Spike Prime Hub, Spike Essential Hub, and Mindstorms Robot Inventor Hub.
Playstation 5 may have to wait until (if) Sony enables BLE in a future update.
Sorry, that must have happened accidentally as I was moving other linked issues around. Rest assured, this is still on our list 😄
So this happened... because the internal preview automatically closed this issue!
:tada: Go check it out! :tada:
FWIU, the 8bitdo USB adapters remap all of the supported controllers to an Xbox controller.
8BitDo controllers and arcade sticks
- Xbox Series X | S, Xbox One Bluetooth controllers
- PS5, PS4, PS4 Pro, PS3 controllers
- Switch Pro, Switch Joy-Con, Wii U Pro, Wiimote
Special Features
- Vibration support on X-input mode ³
- 6-axis motion on Switch X-input, D-input, Mac mode, Switch mode Upgradable firmware
- Virtually lag-free
Steam Input also remaps to what looks like an Xbox controller FWICS.
How much work is there to add additional BLE GATT HIDs so that other controllers work?
On Fri, Feb 16, 2024, 7:25 AM laurensvalk @.***> wrote:
If someone can sort this out and support both Xbox series, and PlayStation 5 controllers at minimum. I would make a donation to both pybricks and the individual developer of $500.00 each. Must be completed no later then end of Q1 2024.
Originally posted by @mozts2005 https://github.com/mozts2005 in #1024 (comment) https://github.com/pybricks/support/issues/1024#issuecomment-1712963131
I hope the current version meets your needs partially! No matter what you decide, I hope you enjoy the new update 🙂
All Xbox series controllers since 2016 are supported. With Technic Hub, Spike Prime Hub, Spike Essential Hub, and Mindstorms Robot Inventor Hub.
Playstation 5 may have to wait until (if) Sony enables BLE in a future update.
Sorry, that must have happened accidentally as I was moving other linked issues around. Rest assured, this is still on our list 😄
So this happened... because the internal preview automatically closed this issue!
🎉 Go check it out! 🎉
[image: preview] https://www.youtube.com/watch?v=fxInp9cutNg
— Reply to this email directly, view it on GitHub https://github.com/pybricks/support/issues/1024#issuecomment-1948294595, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAMNSYPM4SKF7AQFLIP7QTYT5F2LAVCNFSM6AAAAAAWQGT2TKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYGI4TINJZGU . You are receiving this because you were mentioned.Message ID: @.***>
We've generalized the BLE code considerably, so I'm quite open to adding a few more gamepads. My preference would be gamepads that are somewhat mainstream and easy enough to get worldwide.
We had some serious challenges getting the Xbox Controller pairing/bonding process to work with the Technic Hub, and it could still be better. So having HID over BLE is no guarantee, but certainly worth a try.
This is a really nice addition to pybricks. Is rumble supported? I think that it would be funny as a "beeping" for driving backwards or as a distance sensor indication that some object is close to the vehicle. Or maybe for some interactive games that use the hubs.
Regarding future supported controllers: one that's probably widely available is the amazon luna controller. It's often on sale for about half the price. Other ones like the google stadia controller or the steam controller are discontinued. Another BLE controller that's pretty new and will be available for a while is the 8bitdo neogeo wireless controller.
Is rumble supported?
Not yet, but it could be. We have also generalized the internal "bluetooth write" function. We'd just need to figure out the right combination of commands to send. Same thing for the light, which should be full RGB.
Is rumble supported?
Not yet, but it could be. We have also generalized the internal "bluetooth write" function. We'd just need to figure out the right combination of commands to send. Same thing for the light, which should be full RGB.
I think that the light is RGB for the elite 2 controller but not for the other xbox controllers. I'm not sure tho. Some of them also have multiple rumble motors in different positions. This might be a bit too much, but in theory it should be possible to control the motors independently
So this happened... because the internal preview automatically closed this issue!
This is amazing news guys, thanks a lot for implementing this! On top of that the code blocks are also a very nice new feature that makes the use case of the Xbox controller even more easy to start using for many people :)
I have tested it and it works well, even though in the beginning I had some problems with the website; it was loading assets very slowly (images, icons, even text), not showing much (I don't think it was my internet connection, I did a speedtest), and on top of that it occasionally went completely blank, unable to recover, had to reload the page. But it seems it got stabilized over time, probably due to caching..
Anyways, the controller works fine. I'd have a few questions/suggestions though.
- It seems currently that it is not supported on the city hub, is that due to HW limitation? Too small memory/processing power?
- It took me some time to remember that it disconnects from the Technic Hub when the Xbox controller is connected, and I loose print commands that I wanted to use for debugging (was thinking that it actually didn't go through to the constructor of the controller as I did not get to the next print). I understand that it's a stability issue, I was wondering if there could be a switch that allows it to stay connected for the cases when it actually works? Or would that be very error prone? I guess it depends on the computer/browser? Though I saw your comment that you might try to make this more stable in the future.. Anyways, just wondering if debug printing could be made work when possible/needed.
- I saw the motor class has accessible parameters for the pid controller. Can you give some hints what parameters I should tune (in which directions) if I wanted to make the steering more aggressive/responsive? I guess it would introduce some jerkiness, but it would be nice to play around with it to see what extremes are. Also, I was wondering if there could be two sets of predefined parameters, one for smooth/slow steering, and one for fast but more jerky steering, and then use linear interpolation between the two extremes to easily tune for a model. For example, the Car class could have that linear interpolation parameter.
- I was also wondering about how speed control should be used properly (in general, not just with the controller). If I understand correctly, in case of speed control, we can set angular speed; however, we cannot easily tell the max possible speed for a motor (it depends on the motor as well as the weight of the model, and also on the terrain), so it is hard to know what to set for 100% speed (for example for the max trigger position). In other words, the scale is undefined. Do you have some idea in mind to make this work properly with a proportional controller? My only idea would be to do some calibration, running the model on a flat surface with max power for a few seconds and then measure the speed to get the scale right.
- As for such calibration, does that need to run every time the hub is turned on? Is the steering calibration stored in the Car class in a runtime variable, or could such calibration values be stored permanently? Besides taking time/space, a potential problem with calibration running every time is that the model may not be ready for it (no space to run, being on rough surface where the steering motor is not strong enough to move the wheels to the extremes, etc).
It seems currently that it is not supported on the city hub, is that due to HW limitation? Too small memory/processing power?
Its Bluetooth chip firmware (separate from the main firmware) does not appear to have certain security features enabled like bonding, which the Xbox Controller requires. Before I started this endeavor, I did not know that the Technic Hub would, either. I went for it anyway, and I'm glad about it now :)
It took me some time to remember that it disconnects from the Technic Hub when the Xbox controller is connected, and I loose print commands that I wanted to use for debugging
We hope to improve this in the future. Would you mind adding your findings to #1462? It would be great to have more data points.
I saw the motor class has accessible parameters for the pid controller. Can you give some hints what parameters I should tune (in which directions) if I wanted to make the steering more aggressive/responsive?
This would probably be worth asking in a different issue for more detail. You could print the current values and then set double the default kp as a starting point.
As for such calibration, does that need to run every time the hub is turned on?
Technically, no. But the Car class is a new addition intended to make steering cars very easy to do, so we just initialize every time. But motors have an option during init to skip the reset. If your steering motor needs to run less than 360, you could also rely on the absolute encoder value. Feel free to make a dedicated issue about this with suggestions and ideas as well. The main thing would be to add some sort of UI for deciding when to calibrate or not. Only the first time after boot? Only if the button is pressed? And so on.
Would you mind adding your findings to https://github.com/pybricks/support/issues/1462?
Sure, I will do that, good to see that I can experiment with staying connected. Will post the results.
I'll create some issues with proposals for the rest then!
One thing I wonder though is how to use the absolute angle of the motor. There is only an 'angle' query method for the motor class as I see, and that one gets the accumulated position, is that right? Is there no way to directly get the absolute position? The only way I see is resetting the accumulated position to the absolute position, and then querying that..
Since support for other controllers comes up frequently, here is a good overview of BLE models.
The list isn't very long, so we'll probably just keep using Xbox.
Since support for other controllers comes up frequently, here is a good overview of BLE models.
The list isn't very long, so we'll probably just keep using Xbox.
It's not complete tho. As far as I know, other BLE controllers are Amazon Luna, 8Bitdo NeoGeo (the other 8Bitdo Controllers are BTC) and multiple mobile ones (Flydigi Apex 2, Steelseries Stratus+, Ipega 9078, Ipega 9083S, Nacon MG-X Pro, Razer Jaiju Mobile, PowerA Moga XP5-A, FlyDigi FeiZhi APEX, Saitake 7007f, asus rog kunai 4, Razer Junglecat, Mad Catz C.T.R.L.R. and more). I think that xbox controllers are by far the most common ones in that list. But there might be cheaper ones.