setup.py gives UnicodeDecodeError

[bastings]

setup.py gives a UnicodeDecodeError in Python 3.4.1 because of line 10:

def read(fname):
   return open(os.path.join(os.path.dirname(__file__), fname)).read()`

It is possible to fix for py2/py3 this by using

from io import open

def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname), encoding='utf-8').read()`

[gwenniger]

I found two errors with newer versions of python3.

First, there is a problem in pyter.py The prefix "ur" is no longer supported in newer versions of python, see also: https://stackoverflow.com/questions/26063899/python-version-3-4-does-not-support-a-ur-prefix Seemingly, by replacing "ur" with "r" in the second argument of some of these "re.sub " calls, we can fix this for newer versions of python.

" ### https://stackoverflow.com/questions/26063899/python-version-3-4-does-not-support-a-ur-prefix ### The following lines are problematic, because "ur" is not supported in new versions of python # s = re.sub(r'\s+', ur' ', s) # スペースの個数正規化 ### Gideon: This line is problematic, gives the error invalid syntax # s = re.sub(r'(\d) ([.,]) (\d)', ur'\1\2\3', s) # 0 . 1 -> 0.1 ### Gideon: This line is problematic, gives the error invalid syntax # s = re.sub(r'(Dr|Jr|Prof|Rev|Gen|Mr|Mt|Mrs|Ms) .', ur'\1.', s) # Mr . -> Mr. ### Gideon: This line is problematic, gives the error invalid syntax

### We fix these by replacing "ur" in the second argument with "r"  
s = re.sub(r'\s+', r' ', s)  # スペースの個数正規化    ### Gideon: Fixed by replacing "ur" by "r" as recommended on stacoverflow
s = re.sub(r'(\d) ([.,]) (\d)', r'\1\2\3', s)  # 0 . 1 -> 0.1    ### Gideon: Fixed by replacing "ur" by "r" as recommended on stacoverflow
s = re.sub(r'(Dr|Jr|Prof|Rev|Gen|Mr|Mt|Mrs|Ms) .', r'\1.', s)  # Mr . -> Mr.

"

In similar spirit, there is also a problem with the function "izip", which does not work also in newer versions of python. It should be replaced with simply "zip" for newer versions. Ideally, the code should be adapted to check for the python version, and have two cases depending on the python version.

https://stackoverflow.com/questions/32261698/my-idle-does-not-recognize-itertools-izip-as-a-function

Replacing "iter.zip" with simply "zip" in pyter/init.py

fixes the problems.