Accelerator keys (ESC and ENTER) don't close the rule editor dialog
Using a JOptionPane for the rule editor dialog would solve this.
Escape cancels auto-complete and enter goes to a new line in the editor.
You mean the combination of both?
I don't think ENTER should close the rule editor. CTRL+ENTER would be nice, though.
Escape should still cancel auto-complete if it's showing, and if it's not then it should close the dialog. If Enter prints a new line in the editor then the dialog should be closed with CTRL+Enter (this would be inline with the rest of the UI).
If you put the editor in a VerifyingOptionPane (or a regular JOptionPane if you don't need the verification aspect), then I think these buttons should work out of the box (providing your auto-completer consumes the KeyPressed event if it's showing).
.... just checked this out and with a JOptionPane it works out of the box i.e. CTRL+Enter if a text area has the focus (and this also has the plus that the button arrangement will be standard for the platform/look&feel).