esp32-snippets
esp32-snippets copied to clipboard
nkolban
ESP32 BT-HID Device
Is it possible to use the ESP32_BLE_Arduino library to build a HID-Keyboard to send keystrokes to another device? My goal: I would like to connect a ESP32 board to a BT device like an iPad and send a space character. I found https://github.com/asterics/esp32_mouse_keyboard but can't figure out how to this running in Arduino IDE. Any help would be appreciated - I found a lot of users asking for a similar feature for the ESP32.
I'm afraid there is nothing yet in the BLE C++ classes that would make your request possible. Maybe a question posted to one of the forums here might give some positive results:
https://esp32.com/index.php
It is my loose understanding that HID support is based on Bluetooth classic (BR/EDR) as opposed to BLE and, so far, these classes only cover BLE functions.
Thank you very much @nkolban. I already found a project for esp-idf for HID and BT but I was hoping for an example for the arduino IDE. I have no experience with IDF and the tool chain and how to integrate those stuff with arduino. Maybe this will come in the future.
Do you have a link to the HID and BT project that uses ESP-IDF? I'd like to file that and have a look to see what it would take to eventually encapsulate that in the C++ libraries. That would be the first step towards an Arduino encapsulation.
The most complete and supported solution seems to be the BTStack port for the esp32 for IDF here https://github.com/bluekitchen/btstack/tree/master/port/esp32 within this library there are various examples for HID devices and other BT applications.
OOOh thats cool. But scary as well. If I get a sense of that story, it is a replacement for the Bluetooth stack supplied by Espressif (which I believe is based on Bluedroid stack implementation).
Yes so this might be a problem to have both stacks available at the same time in an Arduino library. The example under https://github.com/asterics/esp32_mouse_keyboard uses the bt.h from the regular Espressif library so (see ble_hidd_demo_main.c).
New day, new lessons:
My esp32 just become HID keyboard. ANd now its HID mouse:
Of course its not real keyboard and mouse, but my windows 10 is stupid and thinks it is ;)
@chegewara Have you build an Arduino project for that already? I would be interested :-) Sending a keyboard key to any BT device like an iPAD would be a great use case for the ESP32.
Oh no, just fooled my laptop by setting appearance(). The rest of code has nothing to do with hid keyboard. But i may do some research and see if this is hard to do.
I have some good news and some bad news. Good news that work is in progress. Bad news is driver crash under windows with error code 0x0a(device cant start). I can say for sure what is the reason, but its a chance that hid keybord requires secure connection and we dont have implemented it yet.
My next step will be to rewrite code in esp-idf and see what happen and maybe add some security there.
EDIT more good news, my samsung s4 connects to esp32 hid keyboard, so now i have to find way to send some text
I have good news. Esp32 hid keyboard is connecting with my android phone and i can send text. This mean hid keyboard works, at least software part. Still is some work to do but we know it can be done.
@chegewara great work, would you share your current project to play around with it? I just need a running starting point for the Arduino IDE :-)
Im not sure yet, i will provide code or share links ive been using during research. Im thinking about writing blog article about this. For now maybe this will help you a bit: https://github.com/I0x0I/DIY-A-BLE-Keyboard
https://docs.mbed.com/docs/ble-hid/en/latest/api/md_doc_HIDService.html
This is very important documentation if you want to play with HID devices. Appendix E gives some good look at what descriptors are required to setup hid device: http://www.usb.org/developers/hidpage/HID1_11.pdf
Services and characteristics:
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<configuration>
<service uuid="1800">
<description>Generic Access Profile</description>
<characteristic uuid="2a00">
<properties read="true" const="true"/>
<value>BGT Keyboard Demo</value>
</characteristic>
<characteristic uuid="2a01">
<properties read="true" const="true"/>
<value type="hex">c103</value>
</characteristic>
</service>
<service type="primary" uuid="180A" id="manufacturer">
<characteristic uuid="2A29">
<properties read="true" const="true"/>
<value>Bluegiga</value>
</characteristic>
<characteristic uuid="2A50">
<!-- PnP ID required by HID Profile -->
<properties read="true" const="true"/>
<value type="hex">014700ffffffff</value>
</characteristic>
</service>
<service uuid="180f">
<description>Battery</description>
<characteristic uuid="2a19" id="xgatt_battery">
<properties read="true"/>
<value type="hex">32</value>
</characteristic>
</service>
<service uuid="1812" advertise="true">
<characteristic uuid="2a4d" id="hid_keyboard_in">
<!-- Keyboard input report -->
<properties notify="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!-- Report reference id=0x00 type=0x01 (input) -->
<properties read="true" const="true"/>
<value type="hex">0001</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4d" id="hid_keyboard_out">
<!-- Keyboard output report -->
<properties write="true" write_no_response="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!-- Report reference id=0x00 type=0x02 (output) -->
<properties read="true" const="true"/>
<value type="hex">0002</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4d" id="hid_keyboard_feature">
<!-- Keyboard feature report -->
<properties write="true" read="true"/>
<value length="20" variable_length="true"/>
<descriptor uuid="2908">
<!-- Report reference id=0x00 type=0x03 (feature) -->
<properties read="true" const="true"/>
<value type="hex">0003</value>
</descriptor>
</characteristic>
<characteristic uuid="2a4b">
<!-- Report map example from USB HID Specification -->
<properties read="true" authenticated_read="true" const="true"/>
<value type="hex">
05010906A101050719E029E71500250175019508810295017508810195057501050819012905910295017503910195067508150025650507190029658100C0
</value>
</characteristic>
<characteristic uuid="2a4a">
<!--
HID Information version=0x0100 countrycode=0x00 flags=0x02 (normally connectable)
-->
<properties read="true" const="true"/>
<value type="hex">00010002</value>
</characteristic>
<characteristic uuid="2a4c" id="hid_control">
<!-- HID Control point, used to control suspending -->
<properties write_no_response="true"/>
<value length="1"/>
</characteristic>
<characteristic uuid="2a22" id="hid_boot_keyboard_in">
<!-- Boot Keyboard input report -->
<properties notify="true" read="true"/>
<value length="20" variable_length="true"/>
</characteristic>
<characteristic uuid="2a32" id="hid_boot_keyboard_out">
<!-- Boot Keyboard output report -->
<properties write_no_response="true" read="true" write="true"/>
<value length="20" variable_length="true"/>
</characteristic>
<characteristic uuid="2a4e" id="hid_protocol_mode">
<!-- Protocol mode select -->
<properties write_no_response="true" read="true"/>
<value length="1"/>
</characteristic>
</service>
</configuration>
@chegewara I managed to run the BlueKitchen ESP32 Port for ESP-IDF (https://github.com/bluekitchen/btstack/blob/master/example/hid_keyboard_demo.c) it runs on my ESP32 board and sends continuously text messages to may mac book and to my android phone as soon as you pair them. The whole setup for the ESP-IDF and libraries is quite complicated so I cannot use this process for teaching. This is why I was looking for a nice wrapper for the Arduino IDE so that you can fire up a example project, hit flash and project runs :-)
Can you wait few days, im still working on arduino code. At the moment im just sending random letter but i still missing one thing. My code requires that notifications to be turned on on client side (my android phone). At the moment only way to achieve this is to open nRF connect, connect to esp32 hid and turn on notifications.
This is what i did so far. There is still a lot to do but it should send random character in loop:
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#include <BLE2902.h>
#include <string>
// See the following for generating UUIDs:
// https://www.uuidgenerator.net/
bool connected = false;
BLEService *pService;
BLEService *pService1;
BLECharacteristic *reportChar1;
BLECharacteristic *reportChar2;
BLECharacteristic *reportChar3;
class MyCallbacks : public BLEServerCallbacks {
void onConnect(BLEServer* pServer){
connected=true;
}
void onDisconnect(BLEServer* pServer){
connected=false;
}
};
void setup() {
Serial.begin(115200);
Serial.println("Starting BLE work!");
BLEDevice::init("ESP32");
BLEServer *pServer = BLEDevice::createServer();
pServer->setCallbacks(new MyCallbacks());
pService = pServer->createService((uint16_t)0x180a);
pService1 = pServer->createService((uint16_t)0x1812, 30);
setupCharacteristics();
pService->start();
pService1->start();
BLEAdvertising *pAdvertising = pServer->getAdvertising();
pAdvertising->setAppearance(961);
pAdvertising->addServiceUUID((uint16_t)0x1812);
pAdvertising->start();
Serial.println("Characteristic defined! Now you can read it in your phone!");
}
uint8_t v[] = {0x0, 0x0, 0x0, 0x0,0x0,0x0,0x0,0x0};
void loop() {
// put your main code here, to run repeatedly:
delay(5000);
if(connected){
uint8_t a[] = {0x0, 0x0, random(0x04,0x26), 0x0,0x0,0x0,0x0,0x0};
reportChar1->setValue(a,sizeof(a));
reportChar1->notify();
reportChar1->setValue(v, sizeof(v));
reportChar1->notify();
}
}
void setupCharacteristics() {
BLECharacteristic *manufacturer = pService->createCharacteristic(
(uint16_t)0x2a29,
BLECharacteristic::PROPERTY_READ
);
std::string name = "espressif";
manufacturer->setValue(name);
BLECharacteristic *pnpIDChar = pService->createCharacteristic(
(uint16_t)0x2a50,
BLECharacteristic::PROPERTY_READ
);
const uint8_t pnp[] = {0x01,0xe5,0x02,0xcd,0xab,0x01,0x00};
pnpIDChar->setValue((uint8_t*)pnp, sizeof(pnp));
BLECharacteristic *hidInfoChar = pService1->createCharacteristic(
(uint16_t)0x2a4a,
BLECharacteristic::PROPERTY_READ
);
const uint8_t val1[] = {0x00,0x01,0x00,0x02};
hidInfoChar->setValue((uint8_t*)val1, 4);
BLECharacteristic *reportMapChar = pService1->createCharacteristic(
(uint16_t)0x2a4b,
BLECharacteristic::PROPERTY_READ
);
const uint8_t val2[] = {
0x05,0x01,0x09,0x06,0xA1,0x01,0x05,0x07,
0x19,0xE0,0x29,0xE7,0x15,0x00,0x25,0x01,
0x75,0x01,0x95,0x08,0x81,0x02,0x95,0x01,
0x75,0x08,0x81,0x01,0x95,0x05,0x75,0x01,
0x05,0x08,0x19,0x01,0x29,0x05,0x91,0x02,
0x95,0x01,0x75,0x03,0x91,0x01,0x95,0x06,
0x75,0x08,0x15,0x00,0x25,0x65,0x05,0x07,
0x19,0x00,0x29,0x65,0x81,0x00,0xC0}; //TODO
reportMapChar->setValue((uint8_t*)val2, sizeof(val2));
BLECharacteristic *hidControlChar = pService1->createCharacteristic(
(uint16_t)0x2a4c,
BLECharacteristic::PROPERTY_WRITE_NR
);
reportChar1 = pService1->createCharacteristic(
(uint16_t)0x2a4d,
BLECharacteristic::PROPERTY_READ |
BLECharacteristic::PROPERTY_NOTIFY
);
BLEDescriptor *desc1 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
const uint8_t desc1_val[] = {0x01, 0};
desc1->setValue((uint8_t*)desc1_val, 1);
reportChar1->addDescriptor(desc1);
reportChar1->addDescriptor(new BLE2902());
reportChar2 = pService1->createCharacteristic(
(uint16_t)0x2a4d,
BLECharacteristic::PROPERTY_READ |
BLECharacteristic::PROPERTY_WRITE
);
BLEDescriptor *desc2 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
const uint8_t desc2_val[] = {0x02, 0};
desc2->setValue((uint8_t*)desc2_val, 1);
reportChar2->addDescriptor(desc2);
reportChar3 = pService1->createCharacteristic(
(uint16_t)0x2a4d,
BLECharacteristic::PROPERTY_READ |
BLECharacteristic::PROPERTY_WRITE |
BLECharacteristic::PROPERTY_WRITE_NR
);
BLEDescriptor *desc3 = new BLEDescriptor(BLEUUID((uint16_t)0x2908));
const uint8_t desc3_val[] = {0x03, 0};
desc3->setValue((uint8_t*)desc3_val, 1);
reportChar3->addDescriptor(desc3);
BLECharacteristic *protocolModeChar = pService1->createCharacteristic(
(uint16_t)0x2a4e,
BLECharacteristic::PROPERTY_WRITE_NR
);
BLECharacteristic *bootInputChar = pService1->createCharacteristic(
(uint16_t)0x2a22,
BLECharacteristic::PROPERTY_NOTIFY
);
bootInputChar->addDescriptor(new BLE2902());
BLECharacteristic *bootOutputChar = pService1->createCharacteristic(
(uint16_t)0x2a32,
BLECharacteristic::PROPERTY_READ |
BLECharacteristic::PROPERTY_WRITE |
BLECharacteristic::PROPERTY_WRITE_NR
);
}
@chegewara Great work!!! It works like a charm on my Android phone. I can also connect the ESP32 with iOS and Mac OS, but no characters appear there - guess this is the security issue problem you were talking about. For testing I used two ESP32 Hellcat LoRa boards. One board is sending a string over LoRa, to the other board which is connected as HID device on Android transferring all input received as keystrokes.
Any news about this? Would be super cool to use it as a AT/PS2 to BLE HID Keyboad adapter.
@ripper121 What news do you need? Its working for few weeks now. There is one example added. If you have questions, issues or suggestions im open and ready to help.
It should work with arduino too. Last time i checked it did work with arduino.
Hi great work, I tried it on my windows 10, but couldn't manage to make it work. First I got a "This device cannot start (Code 10)" and fixed it with https://www.kapilarya.com/fix-this-device-cannot-start-code-10-in-windows-10
After that I've got: "Driver error" - It connects and pairs ok, but doesn't work.
Monitor log: Starting BLE work! Characteristic defined! Now you can read it in your phone! [0;31mE (27800) BT: lmp_version_below LMP version 6 < 8[0m [0;31mE (27860) BT: Call back not found for application conn_id=3[0m [0;31mE (33978) BT: bta_gattc_conn_cback() - cif=3 connected=0 conn_id=3 reason=0x0013[0m
any hint? thanks
Hi, most likely, and im only guessing right now, its problem with report map. Its this part of code (example code):
const uint8_t reportMap[] = {
USAGE_PAGE(1), 0x01, // Generic Desktop Ctrls
USAGE(1), 0x06, // Keyboard
COLLECTION(1), 0x01, // Application
REPORT_ID(1), 0x01, // REPORTID
USAGE_PAGE(1), 0x07, // Kbrd/Keypad
USAGE_MINIMUM(1), 0xE0,
USAGE_MAXIMUM(1), 0xE7,
LOGICAL_MINIMUM(1), 0x00,
LOGICAL_MAXIMUM(1), 0x01,
REPORT_SIZE(1), 0x01, // 1 byte (Modifier)
REPORT_COUNT(1), 0x08,
INPUT(1), 0x02, // Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position
REPORT_COUNT(1), 0x01, // 1 byte (Reserved)
REPORT_SIZE(1), 0x08,
INPUT(1), 0x01, // Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position
REPORT_COUNT(1), 0x05, // 5 bits (Num lock, Caps lock, Scroll lock, Compose, Kana)
REPORT_SIZE(1), 0x01,
USAGE_PAGE(1), 0x08, // LEDs
USAGE_MINIMUM(1), 0x01, // Num Lock
USAGE_MAXIMUM(1), 0x05, // Kana
OUTPUT(1), 0x02, // Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile
REPORT_COUNT(1), 0x01, // 3 bits (Padding)
REPORT_SIZE(1), 0x03,
OUTPUT(1), 0x01, // Const,Array,Abs,No Wrap,Linear,Preferred State,No Null Position,Non-volatile
REPORT_COUNT(1), 0x06, // 6 bytes (Keys)
REPORT_SIZE(1), 0x08,
LOGICAL_MINIMUM(1), 0x00,
LOGICAL_MAXIMUM(1), 0x65, // 101 keys
USAGE_PAGE(1), 0x07, // Kbrd/Keypad
USAGE_MINIMUM(1), 0x00,
USAGE_MAXIMUM(1), 0x65,
INPUT(1), 0x00, // Data,Array,Abs,No Wrap,Linear,Preferred State,No Null Position
END_COLLECTION(0)
};
btw, ive been testing hid examples with windows 10 and there is no need to edit registers to make it works
If you can post report map i will try to check it, also screenshot with error would be great
@hmatulis To make sure that nothing has been changed in esp-idf that could have broke our code i just pulled latest esp-idf and cpp_utils, compiled HID keyboard sample and flashed it. All works fine on android phone and on my windows 10 laptop.
PS One thing, there is small change in esp-idf structure, which was forced by me on devs and they finally changed code, but its not related to your primary issue. This can cause compilation errors, but i have already fixed it and posted PR.
@chegewara I think I have the same problem, literally copy pasted your code, put it on an ESP32 geekworm devboard. Windows and Android recognize it as HID device, but it doesn't 'type' anything...
Could you guys test this bin file? To use it you need to remove .txt from name.
From first scrennshoot i can guess its something with this report map const uint8_t reportMap[].
Could you use this website to parse report map and send result?
http://eleccelerator.com/usbdescreqparser/