kbd-audio
kbd-audio copied to clipboard
ggerganov
Feedback
I was a bit surprised by the huge amount of interest that this project generated. I had no idea if keytap works reliably or not, so I am happy to see that people actually like it.
I am still curious to know how reliable it is. For example, I haven't been able to get any good prediction results when using non-mechanical keyboards. Would be interesting to know if this is the case for everyone.
Feel free to share your experience here.
Hey man love the work you done with this. I was wondering can it be used with a microphone plugged into the laptop as opposed to the built in microphone in the laptop?
Cheers
In addition what setup did you use and how much machine learning did you do to get your results?
The capturing tools do support an option to select the microphone to use (i.e. the capture device).
You simply need to provide the command line argument -cN where you replace N with the capture device ID.
For example, on my machine, when I run the record tool I get this info printed:
$ ./record test.kbd
Usage: ./record output.kbd [-cN]
-cN - select capture device N
Recording 5 frames per key press
Found 3 capture devices:
- Capture device #0: 'Built-in Microphone'
- Capture device #1: 'Soundflower (2ch)'
- Capture device #2: 'Soundflower (64ch)'
Attempt to open capture device 0 : 'Built-in Microphone' ...
Opened capture device succesfully!
Frequency: 24000
Format: 33056 (4 bytes)
Channels: 1
Samples: 512
You can see the 3 available capture devices listed in the standard output. By default, the tools select the capture device with ID = 0. For example, if I wanted to use capture device 1 instead, then I would do:
$ ./record test.kbd -c1
Usage: ./record output.kbd [-cN]
-cN - select capture device N
Recording 5 frames per key press
Found 3 capture devices:
- Capture device #0: 'Built-in Microphone'
- Capture device #1: 'Soundflower (2ch)'
- Capture device #2: 'Soundflower (64ch)'
Attempt to open capture device 1 : 'Soundflower (2ch)' ...
Opened capture device succesfully!
Frequency: 24000
Format: 33056 (4 bytes)
Channels: 1
Samples: 512
Regarding the machine learning question: On my iMac, using the build-in mic, I can record just 2-3 keytaps of each key and be able to recognise the key audio with very high accuracy (as demonstrated in the video). The result really depends on the setup you have - mic, keyboard, surroundings, etc. More details about the approach are in the blog post.
Thank you for the quick response.
I tried using the built in microphone but I seem to get a very low recall rate on my laptop. I don’t know if you have any suggestions?
Also one other thing was the use of command line ./keytap-gui input0.kbd, when trying to open the command line states there isn’t a input0.kbd file? I replace it with output.kbd but the recall rate inst of that on the video.
Any suggestions on how to help would be brilliant.
On 2 Apr 2019, at 22:42, Georgi Gerganov <[email protected]mailto:[email protected]> wrote:
The capturing tools do support an option to select the microphone to use (i.e. the capture device). You simply need to provide the command line argument -cN where you replace N with the capture device ID.
For example, on my machine, when I run the record tool I get this info printed:
$ ./record test.kbd Usage: ./record output.kbd [-cN] -cN - select capture device N
Recording 5 frames per key press Found 3 capture devices: - Capture device #0: 'Built-in Microphone' - Capture device #1: 'Soundflower (2ch)' - Capture device #2: 'Soundflower (64ch)' Attempt to open capture device 0 : 'Built-in Microphone' ... Opened capture device succesfully! Frequency: 24000 Format: 33056 (4 bytes) Channels: 1 Samples: 512
You can see the 3 available capture devices listed in the standard output. By default, the tools select the capture device with ID = 0. For example, if I wanted to use capture device 1 instead, then I would do:
$ ./record test.kbd -c1 Usage: ./record output.kbd [-cN] -cN - select capture device N
Recording 5 frames per key press Found 3 capture devices: - Capture device #0: 'Built-in Microphone' - Capture device #1: 'Soundflower (2ch)' - Capture device #2: 'Soundflower (64ch)' Attempt to open capture device 1 : 'Soundflower (2ch)' ... Opened capture device succesfully! Frequency: 24000 Format: 33056 (4 bytes) Channels: 1 Samples: 512
Regarding the machine learning question: On my iMac, using the build-in mic, I can record just 2-3 keytaps of each key and be able to recognise the key audio with very high accuracy (as demonstrated in the videohttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2OjzI9m7W10&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311686403&sdata=5WO1tFBSb7%2BhfTNoHISmhSo%2FJP2Rk4ME6qonWoVFfuE%3D&reserved=0. The result really depends on the setup you have - mic, keyboard, surroundings, etc. More details about the approach are in the blog posthttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fggerganov.github.io%2Fjekyll%2Fupdate%2F2018%2F11%2F30%2Fkeytap-description-and-thoughts.html&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311696412&sdata=yB0nuqTyUzT1PQNJ7rldzinPIJoBTs3It5mq8OTO0vk%3D&reserved=0.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fggerganov%2Fkbd-audio%2Fissues%2F3%23issuecomment-479216304&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311706421&sdata=RPEwHxSDIUMbq13xH%2BweoQbBtTQ7HMAV3df87MY8Uuk%3D&reserved=0, or mute the threadhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAuuk4EuKRc71lChUsQe5r-sWn7AqZO0kks5vc86vgaJpZM4Y6BqR&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311706421&sdata=0hJ97GxMGIAYrEq2j4lby9BdIXLImXf%2FR5%2BskFl9or0%3D&reserved=0.
Also how do you open the application in google?
On 2 Apr 2019, at 22:42, Georgi Gerganov <[email protected]mailto:[email protected]> wrote:
The capturing tools do support an option to select the microphone to use (i.e. the capture device). You simply need to provide the command line argument -cN where you replace N with the capture device ID.
For example, on my machine, when I run the record tool I get this info printed:
$ ./record test.kbd Usage: ./record output.kbd [-cN] -cN - select capture device N
Recording 5 frames per key press Found 3 capture devices: - Capture device #0: 'Built-in Microphone' - Capture device #1: 'Soundflower (2ch)' - Capture device #2: 'Soundflower (64ch)' Attempt to open capture device 0 : 'Built-in Microphone' ... Opened capture device succesfully! Frequency: 24000 Format: 33056 (4 bytes) Channels: 1 Samples: 512
You can see the 3 available capture devices listed in the standard output. By default, the tools select the capture device with ID = 0. For example, if I wanted to use capture device 1 instead, then I would do:
$ ./record test.kbd -c1 Usage: ./record output.kbd [-cN] -cN - select capture device N
Recording 5 frames per key press Found 3 capture devices: - Capture device #0: 'Built-in Microphone' - Capture device #1: 'Soundflower (2ch)' - Capture device #2: 'Soundflower (64ch)' Attempt to open capture device 1 : 'Soundflower (2ch)' ... Opened capture device succesfully! Frequency: 24000 Format: 33056 (4 bytes) Channels: 1 Samples: 512
Regarding the machine learning question: On my iMac, using the build-in mic, I can record just 2-3 keytaps of each key and be able to recognise the key audio with very high accuracy (as demonstrated in the videohttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2OjzI9m7W10&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311686403&sdata=5WO1tFBSb7%2BhfTNoHISmhSo%2FJP2Rk4ME6qonWoVFfuE%3D&reserved=0. The result really depends on the setup you have - mic, keyboard, surroundings, etc. More details about the approach are in the blog posthttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fggerganov.github.io%2Fjekyll%2Fupdate%2F2018%2F11%2F30%2Fkeytap-description-and-thoughts.html&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311696412&sdata=yB0nuqTyUzT1PQNJ7rldzinPIJoBTs3It5mq8OTO0vk%3D&reserved=0.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fggerganov%2Fkbd-audio%2Fissues%2F3%23issuecomment-479216304&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311706421&sdata=RPEwHxSDIUMbq13xH%2BweoQbBtTQ7HMAV3df87MY8Uuk%3D&reserved=0, or mute the threadhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAuuk4EuKRc71lChUsQe5r-sWn7AqZO0kks5vc86vgaJpZM4Y6BqR&data=02%7C01%7Cs4910341%40bournemouth.ac.uk%7C6aeccd5b908a44d1d63508d6b7b40ee3%7Cede29655d09742e4bbb5f38d427fbfb8%7C0%7C0%7C636898381311706421&sdata=0hJ97GxMGIAYrEq2j4lby9BdIXLImXf%2FR5%2BskFl9or0%3D&reserved=0.
Laptop keyboards usually have very bad recall rate - the main reason is that the sound of the keys is very low. I myself have had a nice recall rate only when plugging a mechanical keyboard to my iMac - the clicks are much louder and distinguishable.
The web-version of the tool is at the following link: https://ggerganov.github.io/jekyll/update/2018/11/24/keytap.html
Not relevant to the software itself: but on ideas of how to circumvent this - go somewhere where there is no microphone and load your passwords onto a HID (like a Teensy 3.0), then have it type the passwords; but for every day typing it wouldn't help as the approximate keystrokes could be reconstructed - this project highlights the importance of securing every device in proximity (a secure PC with a compromised phone microphone nearby, for instance). I'd known this was possible for a number of years, but the accuracy with such a small training dataset is totally unnerving - thanks!
