lexima.vim causes <CR> to insert newline when insert completion popup menu is in "state 2"

[gotgenes]

Problem Summary

<CR> both makes a selection and inserts a newline character when the insertion completion popup menu is in "state 2." From :help ins-completion-menu:

There are three states:
1. A complete match has been inserted, e.g., after using CTRL-N or CTRL-P.
2. A cursor key has been used to select another match.  The match was not
   inserted then, only the entry in the popup menu is highlighted.
3. Only part of a match has been inserted and characters were typed or the
   backspace key was used.  The list of matches was then adjusted for what is
   in front of the cursor.

To get to state 2. first bring up the insert completion popup menu with <C-P>, then use the up or down arrows to select a different entry. If an entry is highlighted but the text in the buffer has not been updated to match that entry, you know you've reached state 2. At this point, a <CR> is supposed to behave exactly like <C-Y>, that is the following actions occur:

1. It selects the entry and put its contents into the buffer
2. It closes the popup menu
3. It leaves the cursor in insert mode immediately following the completed word; no newline is inserted

When lexima.vim is present in the runtimepath, a newline is inserted, which is not the expected behavior for state 2.

Example VIMRC

set nocompatible
set hidden

set runtimepath+=~/.vim/plugins/repos/github.com/Shougo/dein.vim

" Required:
call dein#begin('~/.vim/plugins')

" Let dein manage dein
call dein#add('Shougo/dein.vim')
call dein#add('cohama/lexima.vim')
call dein#end()

filetype plugin indent on
syntax enable

if dein#check_install()
  call dein#install()
endif

Environment

OS

OS X 10.11 El Capitan

Vim version

MacVim

macvim: stable 8.0-114 (bottled), HEAD
GUI for vim, made for macOS
https://github.com/macvim-dev/macvim
/usr/local/Cellar/macvim/8.0-113 (2,102 files, 33.5M)
  Built from source on 2016-10-26 at 20:13:15 with: --with-python3 --without-python --with-lua --with-luajit
/usr/local/Cellar/macvim/8.0-114 (2,107 files, 33.6M) *
  Built from source on 2016-10-30 at 11:10:06 with: --with-python3 --without-python --with-lua --with-luajit
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/macvim.rb
==> Dependencies
Recommended: cscope
Optional: lua, luajit
==> Requirements
Build: xcode
Recommended: python
Optional: python3
==> Options
--with-custom-python
	Build with a custom Python 2 instead of the Homebrew version.
--with-lua
	Build with lua support
--with-luajit
	Build with luajit support
--with-override-system-vim
	Override system vim
--with-python3
	Build with python3 support
--without-cscope
	Build without cscope support
--without-python
	Build without python support
--HEAD
	Install HEAD version
==> Caveats
.app bundles were installed.
Run `brew linkapps macvim` to symlink these to /Applications.

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Oct 30 2016 11:09:57)
MacOS X (unix) version
Included patches: 1-52
Compiled by Homebrew
Huge version with MacVim GUI.  Features included (+) or not (-):
+acl             +find_in_path    -mouse_sysmouse  -tag_any_white
+arabic          +float           +mouse_urxvt     +tcl
+autocmd         +folding         +mouse_xterm     +termguicolors
+balloon_eval    -footer          +multi_byte      +terminfo
+browse          +fork()          +multi_lang      +termresponse
++builtin_terms  +fullscreen      -mzscheme        +textobjects
+byte_offset     -gettext         +netbeans_intg   +timers
+channel         -hangul_input    +num64           +title
+cindent         +iconv           +odbeditor       +toolbar
+clientserver    +insert_expand   +packages        +transparency
+clipboard       +job             +path_extra      +user_commands
+cmdline_compl   +jumplist        +perl            +vertsplit
+cmdline_hist    +keymap          +persistent_undo +virtualedit
+cmdline_info    +lambda          +postscript      +visual
+comments        +langmap         +printer         +visualextra
+conceal         +libcall         +profile         +viminfo
+cryptv          +linebreak       -python          +vreplace
+cscope          +lispindent      +python3         +wildignore
+cursorbind      +listcmds        +quickfix        +wildmenu
+cursorshape     +localmap        +reltime         +windows
+dialog_con_gui  +lua             +rightleft       +writebackup
+diff            +menu            +ruby            -X11
+digraphs        +mksession       +scrollbind      -xfontset
+dnd             +modify_fname    +signs           +xim
-ebcdic          +mouse           +smartindent     -xpm
+emacs_tags      +mouseshape      +startuptime     -xsmp
+eval            +mouse_dec       +statusline      -xterm_clipboard
+ex_extra        -mouse_gpm       -sun_workshop    -xterm_save
+extra_search    -mouse_jsbterm   +syntax          
+farsi           +mouse_netterm   +tag_binary      
+file_in_path    +mouse_sgr       +tag_old_static  
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/Applications/MacVim.app/Contents/Resources/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe  -DMACOS_X_UNIX  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: clang   -L. -L/usr/local/lib -L. -L/usr/local/lib  -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon       -lm  -lncurses -liconv -framework Cocoa  -pagezero_size 10000 -image_base 100000000 -L/usr/local/opt/luajit/lib -lluajit-5.1 -fstack-protector  -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE -lperl  -L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/config-3.5m -lpython3.5m -framework CoreFoundation -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -framework Ruby    

Plugin version

This is on commit 0aa05cb

Steps to reproduce

1. Edit a new file with the following contents:

   ab
   abcd
   abcdef
   

2. In insert mode, on a new line, type a, then activate keyword completion with <C-P> (state 1.)

3. Push the Up arrow to highlight a different entry (state 2.)

4. Hit Enter key to issue a <CR>

5. Observe that the entry was selected and used, but that a newline was entered and the cursor is now on the line below the completion

Retry the following steps without lexima.vim in the runtimepath and you will observe in step 5 that no newline is inserted and that the cursor remains immediately after the insertion.

[sh8]

+1

[cohama]

Thanks for reporing.

In the past, I introduced some codes to hide popup menu forcely because some auto completion plugins (e.g. neocomplete) conflict with lexima.vim.

These codes maybe cause this issue.

Temporarily you can disable that feature by adding the following code into your vimrc.

let g:lexima_no_default_rules = 1
call lexima#set_default_rules()
call lexima#insmode#map_hook('before', '<CR>', '')

If you use any auto completion plugin, <CR> conflicts lexima's one. It requires more time to solve both 2 problems simultaneously.

[cohama]

BTW, I don't use cursor keys, so that the completion state is not to be 2. I usually use <C-p>/<C-n>, that changes not only candidate but also inserted text.

[gotgenes]

@cohama To be fair, I only discovered the different popup menu states yesterday. I was reading the Vim documentation and trying to figure out what was happening. I was confused when I couldn't get state 2 to activate. When I removed all my plugins I found that state 2 would finally activate with the arrow keys, and then I traced the lack of activation of state 2 down to lexima.

I also rarely use the arrow keys, and like I said, I did not know about the different popup states until yesterday, but I thought I would report it, since it does mean lexima changes default Vim behavior. I just wanted to make the project aware of this.

Thank you for providing lexima!

[docwhat]

There is a similar problem with deoplete in Neovim. Pressing Enter just closes the popup but doesn't insert a newline. This was driving me crazy, but I tracked it down to:

:verbose imap <CR>
i  <CR>        * pumvisible() ? "\<C-Y>" : lexima#expand('<CR>', 'i')
        Last set from ~/.local/share/nvim/plugged/lexima.vim/autoload/lexima/insmode.vim

I tried to override it with:

inoremap <silent> <CR> <C-r>=<SID>my_cr_function()<CR>
function! s:my_cr_function() abort
  " Combine deoplete's smart close with lexima's expansion.
  return pumvisible() ? deoplete#smart_close_popup() . "\<CR>" : lexima#expand('<CR>', 'i')
endfunction

But this would still be overridden by the lexima value.

Using your code snippet before my remap fixed it, but I'm not sure what it is doing. What features am I losing out on?

let g:lexima_no_default_rules = 1
call lexima#set_default_rules()
call lexima#insmode#map_hook('before', '<CR>', '')

My workaround.

[rainerborene]

@docwhat I had the same problem. I'm using this workaround instead:

let g:lexima_no_default_rules = 1
call lexima#set_default_rules()
call lexima#insmode#map_hook('before', '<CR>', '')

function! s:my_cr_function() abort
  return deoplete#close_popup() . lexima#expand('<CR>', 'i')
endfunction

inoremap <expr> <cr> <sid>my_cr_function()