strip-hints fails on files without line end at the end on ubuntu python 3.6.7

[muntyan]

Repro: create a file with a single # in it, no line ends, strip-hints it. See below for the error. This happens only on Ubuntu 18.04.2 LTS for me, it works fine on windows and on centos with python 3.6.5. Let me know if I can help debug it, I don't know where to even start.

(venv3) ymuntyan@rbrunel-lx:~/projects/hyper-api$ python --version
Python 3.6.7
(venv3) ymuntyan@rbrunel-lx:~/projects/hyper-api$ cat /tmp/file.py
#(venv3) ymuntyan@rbrunel-lx:~/projects/hyper-api$ strip-hints /tmp/file.py
Traceback (most recent call last):
  File "/home/local/TSI/ymuntyan/projects/hyper-api/build/release_eapi-py.library/venv3/bin/strip-hints", line 11, in <module>
    load_entry_point('strip-hints==0.1.4', 'console_scripts', 'strip-hints')()
  File "/home/local/TSI/ymuntyan/projects/hyper-api/build/release_eapi-py.library/venv3/lib/python3.6/site-packages/strip_hints/strip_hints_main.py", line 441, in process_command_line
    only_assigns_and_defs, only_test_for_changes)
  File "/home/local/TSI/ymuntyan/projects/hyper-api/build/release_eapi-py.library/venv3/lib/python3.6/site-packages/strip_hints/strip_hints_main.py", line 366, in strip_file_to_string
    processed_code = stripper.strip_type_hints_from_file(filename)
  File "/home/local/TSI/ymuntyan/projects/hyper-api/build/release_eapi-py.library/venv3/lib/python3.6/site-packages/strip_hints/strip_hints_main.py", line 345, in strip_type_hints_from_file
    result = tokens.untokenize()
  File "/home/local/TSI/ymuntyan/projects/hyper-api/build/release_eapi-py.library/venv3/lib/python3.6/site-packages/strip_hints/token_list.py", line 232, in untokenize
    result = tokenize.untokenize(token_tuples)
  File "/usr/lib/python3.6/tokenize.py", line 338, in untokenize
    out = ut.untokenize(iterable)
  File "/usr/lib/python3.6/tokenize.py", line 272, in untokenize
    self.add_whitespace(start)
  File "/usr/lib/python3.6/tokenize.py", line 234, in add_whitespace
    .format(row, col, self.prev_row, self.prev_col))
ValueError: start (1,1) precedes previous end (2,0)

[abarker]

This is actually a bug which was introduced in the untokenize routine in the Python library. See here: https://bugs.python.org/issue35107. Strip-hints uses that routine to convert modified token lists back to text code files.

The current suggested workaround is to basically add a newline to the end of files which are missing them. Getting and testing the last character can be done, for example, with the code here. Just run last_char = get_last_utf8_char(filename, ignore_newlines=False) and test last_char == "\n".