mathlive icon indicating copy to clipboard operation
mathlive copied to clipboard

Any way to programatically give key input events?

Open oatymart opened this issue 4 years ago • 7 comments

I'm trying to set up an end-to-end test using Cypress. I can already render a Mathfield and simulate some user input to it by triggering clicks on virtual keyboard buttons.

What I'd like to do now is simulate user keypresses. Taking Cypress out of the equation for a moment, here's my non-working code executed in the browser console.

let elt = $('math-field'); // ok
elt.focus(); // ok, document.activeElement changed
document.dispatchEvent(new KeyboardEvent('keydown', { key:'e', keyCode: 69 }));
document.dispatchEvent(new KeyboardEvent('keyup', { key:'e', keyCode: 69 })); // nada

Am I sending wrong events or targeting the wrong element? Is this possible to do, or should I just use the elt.setValue() method, even if my users won't use that?

oatymart avatar Feb 17 '21 15:02 oatymart

That's a bit of a complicated situation. The short answer is "no, but...".

AFAIK there are a couple of things that make this difficult/impossible:

  • browsers don't dispatch keyboard events that have not been initiated by the user (see the isTrusted event flag). If they would, a malicious web page could quit your browser when you visit it, or worse.
  • keyboard events are fiendishly complicated, they never occur in isolation, but are a sequence of carefully orchestrated events (keydown, keyup, but also keypress, compositionStart, compositionupdate, etc..). So they would be difficult to do 'correctly' programmatically anyway.

The best alternative I can offer is the insert() method, which will insert a character. It's a bit limited, because you can't simulate thing like typing a keyboard shortcut, but it's something. If there is interest, I could look into adding a method that would take a keyboard event as input and more completely simulate keyboard interaction, though. Let me know what you think.

arnog avatar Feb 17 '21 18:02 arnog

Thanks for the reply. For now, the insert() method will get me through with only a small part of the user flow missing. I was hoping to take advantage of Cypress's cy.get(element).focus().type(characters) commands to abstract away the difficulty of simulating key events, but I could not get it to work, hence the manual approach.

If I find anything new I'll update this.

oatymart avatar Feb 18 '21 09:02 oatymart

I managed to get it working using Protractor and the main issue is related to the built-in hidden textarea element, which is the one you need for sending keyboard events. It's a bit of a hack, but it can provide a direction of the solution, as I added the following code to my component's implementation - this.mathField.shadowRoot.innerHTML += "<style>" + "textarea {" + "width: 10px !important;" + "height: 10px !important;" + "background: transparent;" + "caret-color: transparent;" + "transform: none !important;" + "}" + "</style>"; (Line breaks got messed up, but I made it look like CSS in Typescript) mathField is a reference to a member created using new MathfieldElement(). Eventually, the textarea element needs to be little "less hidden" in order to use it in E2E tests. In addition, it is being created inside a shadow-root element, so there's another challenge in getting to it, but I managed to overcome that too (document.querySelector("math-field").shadowRoot and then finding by css ".ML__textarea__textarea" inside it).

evyatarbh avatar Apr 04 '21 11:04 evyatarbh

Would the option substituteTextArea be useful at all? It's an option that can be used to replace the standard hidden <textarea> by markup of your choice. This would allow you to avoid injecting the <style> tag, at least.

arnog avatar Apr 04 '21 12:04 arnog

@arnog I tried, but couldn't get that to work. In addition, note that I wrapped MathLive in an Angular 8 component, so implementation was somewhat unique I suppose.

evyatarbh avatar Apr 04 '21 12:04 evyatarbh

🙁

arnog avatar Apr 04 '21 12:04 arnog

@arnog Anyway, what I was trying to suggest is that the built-in textarea element would be "less hidden" by applying the styles I used in my previous comment. It would still will not be useable by the standard user, but automatic testing frameworks would be able to interact with it. As a temporary workaround it can be applied the way I did it, but obviously it would be better to be defined that way in the component itself (without all those !important statements I had to use)

evyatarbh avatar Apr 04 '21 13:04 evyatarbh

I've added a keyboard-sink part to the keyboard sink element. Accessible via math-field:part(keyboard-sink).

arnog avatar Mar 14 '23 02:03 arnog