YAPF clears the buffer on formatting

[AdityaTaggar05]

Describe the bug I have set yapf as my default formatter in .vimrc and whenever I save a .py file, it clears the buffer. And if I change it to black or autopep8, it doesn't.

To Reproduce Minimal vimrc:

Plug 'google/vim-maktaba'
Plug 'google/vim-codefmt'
Plug 'google/vim-glaive'

call plug#end()
call glaive#Install()

augroup autoformat_settings
    autocmd FileType bzl AutoFormatBuffer buildifier
    autocmd FileType c,cpp,proto,javascript,arduino AutoFormatBuffer clang-format
    autocmd FileType dart AutoFormatBuffer dartfmt
    autocmd FileType go AutoFormatBuffer gofmt
    autocmd FileType gn AutoFormatBuffer gn
    autocmd FileType html,css,sass,scss,less,json AutoFormatBuffer js-beautify
    autocmd FileType java AutoFormatBuffer google-java-format
    autocmd FileType python AutoFormatBuffer black
    autocmd FileType rust AutoFormatBuffer rustfmt
    autocmd FileType vue AutoFormatBuffer prettier
augroup END

OS (version) Windows 10 NVIM v0.4.3 and vim-plug yapf 0.30.0

[dbarnett]

Sounds like a dupe of #61. It would help to know what error yapf is actually reporting if you try running it manually.

[AdityaTaggar05]

@dbarnett Thanks for replying but when I run yapf manually it doesn't give an error.

I have two files, right.py and wrong.py. right.py has no syntax error or runtime errors and wrong.py has one. The outputs for both the files are below:

Running yapf right.py gives:

import re

inp = "AKICKSTARTROUNDOFGOOGLEKICKKICKSTART"

print(re.findall(r"^(KICK)\w*(START)"))

Running yapf wrong.py gives:

Traceback (most recent call last):
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\pytree_utils.py", line 115, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\driver.py", line 104, in parse_string
    return self.parse_tokens(tokens, debug)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\driver.py", line 72, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\parse.py", line 116, in addtoken
    ilabel = self.classify(type, value, context)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\parse.py", line 172, in classify
    raise ParseError("bad token", type, value, context)
lib2to3.pgen2.parse.ParseError: bad token: type=58, value=' ', context=('', (3, 5))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\pytree_utils.py", line 121, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\driver.py", line 104, in parse_string
    return self.parse_tokens(tokens, debug)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\driver.py", line 72, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\parse.py", line 116, in addtoken
    ilabel = self.classify(type, value, context)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\lib2to3\pgen2\parse.py", line 172, in classify
    raise ParseError("bad token", type, value, context)
lib2to3.pgen2.parse.ParseError: bad token: type=58, value=' ', context=('', (3, 5))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\asus\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\ASUS\AppData\Local\Programs\Python\Python37\Scripts\yapf.exe\__main__.py", line 7, in <module>
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\__init__.py", line 365, in run_main
    sys.exit(main(sys.argv))
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\__init__.py", line 135, in main
    verbose=args.verbose)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\__init__.py", line 204, in FormatFiles
    in_place, print_diff, verify, quiet, verbose)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\__init__.py", line 233, in _FormatFile
    logger=logging.warning)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\yapf_api.py", line 100, in FormatFile
    verify=verify)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\yapf_api.py", line 147, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\pytree_utils.py", line 127, in ParseCodeToTree
    raise e
  File "c:\users\asus\appdata\local\programs\python\python37\lib\site-packages\yapf\yapflib\pytree_utils.py", line 125, in ParseCodeToTree
    ast.parse(code)
  File "c:\users\asus\appdata\local\programs\python\python37\lib\ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "test.py", line 3
    inp = "AKICKSTARTROUNDOFGOOGLEKICKKICKSTART
                                              ^
SyntaxError: EOL while scanning string literal

But if I try to run :FormatCode yapf inside nvim, both of the files are emptied.

Can you guide me on what I am doing wrong or what's wrong?

[dbarnett]

I'm not seeing much reason that would explain it. Possibly some quirk with DOS-style line endings if it's Windows-specific? Can you confirm exit codes and what output is going specifically to stdout/stderr in those cases?

I tried with file contents like those (on Linux) and didn't notice any unexpected behavior. I think I was using a slightly older version of yapf, but all that should matter from yapf is (exit code, stdout, stderr).

[AdityaTaggar05]

The exit code is 0

Running yapf right.py returns

import re

inp = "AKICKSTARTROUNDOFGOOGLEKICKKICKSTART"

print(re.findall(r"^(KICK)\w*(START)"))

And the stderr is empty

[dbarnett]

Can you update to d418de0 if you haven't yet and confirm if your symptoms change? Have you tried any other formatters besides yapf?

[cybardev]

Have you tried any other formatters besides yapf?

In their original question they mentioned:

And if I change it to black or autopep8, it doesn't (clear the buffer).

And in their minimal vimrc, black can be seen, so I suppose they have tried other formatters.

[dbarnett]

Fair point. I just have no explanation for why this exit code/stdout/stderr would blank out the buffer from looking at the code, so I was exploring for some systemic issue, maybe something platform-dependent. The two hypotheses I can come up with are

1. Somehow their yapf invocations from inside vim are giving different results than calling directly on the cli.
2. Some very intricate issue (like something related to DOS-style line endings) is causing the buffer to be blanked out for the given output.

Happy to troubleshoot further if someone can reproduce. Otherwise I might tentatively close it and assume d418de0 helped.