str() of Chameleon Exception throws UnicodeDecodeError

[wlang42]

In Zope PageTemplates there is a handy method called "find_bad_templates", which does the obvious: traverse through all PageTemplates in a Folder and listing those, where the "_cook" method fails.

However, we encountered a PageTemplate where the traversing stops with a "UnicodeDecodeError". The reason is that described method catches the Chameleon Exceptions and saves them as a string. Schematically:

def _cook(self):
    ...
    try:
        ..._cook()
    except:
        etype, e = sys.exc_info()[:2]
        self._v_errors = [
            "Compilation failed",
            "%s.%s: %s" % (etype.__module__, etype.__name__, e)
    #               ^^                                       ^
            ]

If an exception occurs, it is converted to a string. And that can fail, if the token contains a non-ascii char. We can reproduce it with Python2 like this:

from chameleon import exc, tokenize
body = u'<p>ein text mit uumlaut\xfc</p>'
string = body[3:-4]
token = tokenize.Token(string, 3, body)
e = exc.LanguageError('Invalid define syntax', token)
str(e)

The last line throws an exception:

Traceback (most recent call last):
  File ...
    str(e)
  File ".../chameleon/src/chameleon/exc.py", line 192, in __str__
    text += "\n" + line
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 47:
  ordinal not in range(128)

I will make a pull request to fix this...