autopilot-deno
autopilot-deno copied to clipboard
Modifier keys don't really seem to work
Description
Modifier keys don't really work like they should on MacOS. Maybe this is only an issue with MacOS Monterey, I don't think it is though. I can't really "describe" the bug in detail because I have no clue what's really going on. All I know is that using toggleKey
and then type
with a single letter really just inserts that letter as text and autopilot will completely ignore that modifier keys are ignored. However, I can compare autopilot's behaviour with cliclick's.
Reproduce
- Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
- Run this:
deno repl --no-check --unstable
- Then enter this and press enter:
import AutoPilot from "https://deno.land/x/autopilot/mod.ts"; const sleep = async (milliseconds) => new Promise(resolve => setTimeout(resolve, milliseconds)); await sleep(5000); await pilot.toggleKey("meta", true); await pilot.toggleKey("shift", true); await pilot.type("g"); await pilot.toggleKey("meta", false); await pilot.toggleKey("shift", false);
- You now have five seconds to switch to your browser again and click on that input field.
- This is the result:
- Install cliclick and run this, then click on the input field on the website again:
cliclick w:5000 kd:cmd,shift t:g ku:cmd,shift
- This is the result:
Expected Behaviour
While modifier keys somehow work individually, they don't work like expected in most cases I experienced. For example, all hotkey combinations that include a modifier key and any ordinary letter, like the typical ⌘+C to copy, just don't work in my experience. To give you one more example, on MacOS, whenever an "open file" dialog is present, you can use ⌘+⇧+G to open a textbox allowing you to enter the path to a folder. Using cliclick for this just works, while autopilot doesn't, and instead just presses G.
Desktop
- OS: MacOS Montery 12.1 Beta (21C5045a)
- Browser Chrome
- Version 0.4.0
Additional context
Going through the issues of this repository, I assume the root of this issue going unnoticed is toggleKey
only allowing specific keys for some reason. Looks like up until version 0.4.0, hotkey stuff did work, by using toggleKey
. Note that I haven't found any evidence of the combination toggleKey(modifier, true); type(letter); toggleKey(modifier, false);
ever working. If one uses ordinary letters with toggleKey
today, everything crashes and rust panicks:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error("unknown variant `c`, expected one of `f1`, `f2`, `f3`, `f4`, `f5`, `f6`, `f7`, `f8`, `f9`, `f10`, `f11`, `f12`, `f13`, `f14`, `f15`, `f16`, `f17`, `f18`, `f19`, `f20`, `f21`, `f22`, `f23`, `f24`, `leftarrow`, `uparrow`, `rightarrow`, `downarrow`, `insert`, `delete`, `home`, `end`, `pageup`, `pagedown`, `tab`, `backspace`, `enter`, `escape`, `space`, `meta`, `alt`, `control`, `shift`, `capslock`", line: 1, column: 10)', src/lib.rs:122:1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Backtrace with RUST_BACKTRACE=1
:
stack backtrace:
0: _rust_begin_unwind
1: core::panicking::panic_fmt
2: core::result::unwrap_failed
3: _toggle_key
4: _ffi_call_unix64
5: _ffi_call_int
6: deno_ffi::ffi_call
7: deno_core::ops_json::op_sync::{{closure}}
8: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
9: deno_core::ops::OpTable::route_op
10: <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn
11: __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
12: __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
13: __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
fatal runtime error: failed to initiate panic, error 5
Backtrace with RUST_BACKTRACE=full
:
stack backtrace:
0: 0x1144675d1 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hec762383110a7685
1: 0x11447f3bb - core::fmt::write::h2d5ecb4b9764759c
2: 0x114465c0a - std::io::Write::write_fmt::h9d7d3ae333151289
3: 0x114468805 - std::panicking::default_hook::{{closure}}::h552de0233eed7dab
4: 0x1144683ef - std::panicking::default_hook::hf4e8e1e5a5c43b90
5: 0x114468ef0 - std::panicking::rust_panic_with_hook::h7c7e0153f3e14d6b
6: 0x11446898e - std::panicking::begin_panic_handler::{{closure}}::h3c7f7ffd2b05c635
7: 0x114467a47 - std::sys_common::backtrace::__rust_end_short_backtrace::h3bdf4f6c89eee6ea
8: 0x1144688fa - _rust_begin_unwind
9: 0x114489d6f - core::panicking::panic_fmt::h88a1b6fbb9084d2c
10: 0x114489e55 - core::result::unwrap_failed::h8abe20a396a5e82a
11: 0x114403a35 - _toggle_key
12: 0x1100571e2 - _ffi_call_unix64
13: 0x110056c35 - _ffi_call_int
14: 0x10f6eff40 - deno_ffi::ffi_call::h7b8d00a17ac9084e
15: 0x10f9613b3 - deno_core::ops_json::op_sync::{{closure}}::h2dff410c79672602
16: 0x10f67b81f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd8d69a3a2de8f014
17: 0x10f67074c - deno_core::ops::OpTable::route_op::h86ef2e6087baaab6
18: 0x10f682a8c - <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn::hfa3b6c5b5cdbaae7
19: 0x1101345d5 - __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
20: 0x110133bc1 - __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
21: 0x11013312e - __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
fatal runtime error: failed to initiate panic, error 5