:py3 command does not work in macvim launching from dock

[hujianxin]

Describe the bug :py3 command works correctly in mvim which launches from terminal, but does not work in mvim launching from dock.

To Reproduce Detailed steps to reproduce the behavior:

1. launch macvim from dock
2. :py3 import subprocess
3. :py3 print( subprocess.Popen('rg ""', shell=True, stdout=subprocess.PIPE).stdout.read() )
4. show nothing

Expected behavior

1. launch macvim from terminal
2. :py3 import subprocess
3. :py3 print( subprocess.Popen('rg ""', shell=True, stdout=subprocess.PIPE).stdout.read() )
4. show the rg output lines

Screenshots The wrong one:

The correct one:

Environment (please complete the following information): Vim version:

• OS: [e.g. Ubuntu 18.04, Windows 10 1809, macOS 10.14]

Additional context I installed macvim by brew cask install macvim

[hujianxin]

➜  ~ /Applications/MacVim.app/Contents/bin/vim  --version
VIM - Vi IMproved 8.1 (2018 May 18, compiled Oct 30 2019 11:57:56)
macOS version
Included patches: 1-2234
Compiled by [email protected]
Huge version with MacVim GUI.  Features included (+) or not (-):
+acl               +file_in_path      +mouse_urxvt       -tcl
+arabic            +find_in_path      +mouse_xterm       +termguicolors
+autocmd           +float             +multi_byte        +terminal
+autochdir         +folding           +multi_lang        +terminfo
-autoservername    -footer            -mzscheme          +termresponse
+balloon_eval      +fork()            +netbeans_intg     +textobjects
+balloon_eval_term +fullscreen        +num64             +textprop
+browse            -gettext           +odbeditor         +timers
++builtin_terms    -hangul_input      +packages          +title
+byte_offset       +iconv             +path_extra        +toolbar
+channel           +insert_expand     +perl/dyn          +transparency
+cindent           +job               +persistent_undo   +user_commands
+clientserver      +jumplist          +postscript        +vartabs
+clipboard         +keymap            +printer           +vertsplit
+cmdline_compl     +lambda            +profile           +virtualedit
+cmdline_hist      +langmap           +python/dyn        +visual
+cmdline_info      +libcall           +python3/dyn       +visualextra
+comments          +linebreak         +quickfix          +viminfo
+conceal           +lispindent        +reltime           +vreplace
+cryptv            +listcmds          +rightleft         +wildignore
+cscope            +localmap          +ruby/dyn          +wildmenu
+cursorbind        +lua/dyn           +scrollbind        +windows
+cursorshape       +menu              +signs             +writebackup
+dialog_con_gui    +mksession         +smartindent       -X11
+diff              +modify_fname      -sound             -xfontset
+digraphs          +mouse             +spell             +xim
+dnd               +mouseshape        +startuptime       -xpm
-ebcdic            +mouse_dec         +statusline        -xsmp
+emacs_tags        -mouse_gpm         -sun_workshop      -xterm_clipboard
+eval              -mouse_jsbterm     +syntax            -xterm_save
+ex_extra          +mouse_netterm     +tag_binary
+extra_search      +mouse_sgr         -tag_old_static
-farsi             -mouse_sysmouse    -tag_any_white
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/Applications/MacVim.app/Contents/Resources/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe  -DMACOS_X -DMACOS_X_DARWIN  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang   -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/[email protected]/lib -L/usr/local/opt/readline/lib -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/[email protected]/lib -L/usr/local/opt/readline/lib  -L/usr/local/lib -o Vim -framework Cocoa -framework Carbon       -lm  -lncurses -liconv -framework AppKit   -fstack-protector  -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE

[hujianxin]

brew install macvim also doesn't work properly

[eirnym]

Could you ensure, that rg program is in your PATH in both cases? You can do it by :echo $PATH in MacVim and if not, edit /etc/paths and/or /etc/paths.d, then login-logout

[hujianxin]

Hi, I can ensure rg is in PATH.

[eirnym]

Could you print out the $PATH from py3 as you run the command from there?

[hujianxin]

Sure! :py3 print(os.environ[PATH'] )

[hujianxin]

@eirnym

[eirnym]

Have you checked both runs, from terminal and from dock?

[hujianxin]

Yes, I have checked the following situations

1. terminal macvim: works
2. terminal nvim: works
3. vimr(gui nvim) lanched from dock: works
4. macvim launched from terminal: works
5. macvim launched from doc: not work

And, not just I met this issue, other guy met it either.

[hujianxin]

Actually, I'm using LeaderF extension which privides Fuzzy file finding function like FZF. this extenssion does not work in macvim launched from dock. So I checked py3 print( subprocess.Popen('rg ""', shell=True, stdout=subprocess.PIPE).stdout.read() ) , and found something different macvim launched from terminal and launched from dock.

[eirnym]

I use this extension as well, but in most cases launch MacVim from terminal.

now when I run MacVim from dock:

• rg '' | cat in terminal works as expected
• :!rg '' works as expected
• if I execute subprocess.Popen('rg ""', shell=True, stdout=subprocess.PIPE).stdout.read()) it will fail as it requires stderr
• if I execute subprocess.Popen('rg ""', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.read()) it will exit with no errors and no output. Probably some environment variables or path or .... I don't know
• if I pass incorrect folder or a folder explicitly, everything works fine as expected.

after a bit of searching and practicing to run it, I've found that this pattern of rg usage is involved: command | rg [OPTIONS] PATTERN. Basically, rg sees that your input is an empty buffer (NOT closed and NOT terminal), so it thinks, that your want to grep the output of previous command which is an empty buffer as it is closed at the beginning when MacVim is running (and most others macOS apps) and subprocess just passes it to rg as it was.

the : passes the stdin as terminal (even a dummy one), so rg works as you expect it.

[eirnym]

You can reproduce it with a simple program below. If stdin is not closed, it will print out a lot of output.

import sys
import subprocess

sys.stdin.close()
print(subprocess.Popen(
    "rg -a '.*'",
    shell=True,
    cwd='/bin/',
    stderr=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stdin=subprocess.DEVNULL)
      .communicate())

[RStankov]

I had the same issue. The fix was to add to my vimrc:

set pythonthreedll=/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/Python
set pythonthreehome=/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/

[ychin]

What version are you using @RStankov ? If you are using the latest version (165), you should use 3.8 instead. MacVim will automatically use the right python DLLs if that's the case.

[RStankov]

I'm on latest 165, this was the Python, I had 🤷‍♂️

Will download 3.8 to see if this will be needed. Thanks :)

[ychin]

Yeah the way Vim works with Python / other scripting languages is by linking against specific libraries. You could technically provide Python 3.7 to Vim by doing what you did in https://github.com/macvim-dev/macvim/issues/1003#issuecomment-674527401 by modifying pythonthreedll, but it could lead to subtle bugs as the API could have changed in small ways, and as such it's ideal to use the same Python version that Vim is linked against.

[RStankov]

@ychin tanks for clarification.

[eirnym]

@hujianxin @RStankov Could you tell if this is still an issue for you?

[RStankov]

@eirnym I still have this issue.

[eirnym]

I see, it works as expected, but stdin is still open for some reason with no input. You should add stdin=subprocess.DEVNULL if you want it to work more portable.

I can't find any information in documentation for ViM, that for :py3 (or any other language) stdin is closed. More, that in ViM's documentation, it's stated, that input is not supported could lead even to crash:

							*python-input*
	Input (via sys.stdin, including input() and raw_input()) is not
	supported, and may cause the program to crash.  This should probably be
	fixed.
`

[eirnym]

Therefore, I threat MacVim's behaviour as a fix for a problem with python input rather than "py3 is not working"

[ychin]

Sorry I haven't been following this issue too much, but after reading up on it and testing it out I agree with @eirnym that you should probably just add stdin=subprocess.DEVNULL to the subprocess call. It's not clear where stdin should come from anyway, and per the documentation it's not really a supported feature.

Also, from playing around with it, when you launch subprocess this way, the stdin that you are using is actually from the terminal. This seems… not ideal because that means the Python code is using your terminal stdin that MacVim is inheriting from. I will probably fix this in the future so even terminal-launched MacVim will not work.'

@hujianxin can you just ask LeaderF to fix their subprocess call to pass a null stdin?

I'm also not sure why VimR works. It may just make a dummy stdin.

[eirnym]

Issue has been fixed in the LeaderF.