Cylc Review: Handle more characters

[wxtim]

Currently Cylc Review cannot handle many characters, including those used to create the Cylc logo on the command line. As a result any log file containing the Cylc Logo will be shown as plain text.

Reproducible Example

# flow.cylc
[scheduling]
    [[graph]]
        R1 = foo

[runtime]
    [[foo]]
        # Ensure that Cylc log has examples of unusable chars
        script = cylc message -- "Āńŷ ōf ţħėşę ŵĭļŀ břēąķ ĈŸļç Rëvîéw"

Navigate to Cylc Review (or start it with cylc review start if you don't have a site deployment)

The job activity log for this task is correctly handled by Cylc Review - most of the other log files are displayed as raw text.

[ColemanTom]

Is Cylc review back, or does it still need Cylc7/py2 as per https://github.com/cylc/cylc-flow/issues/5937 ?

[hjoliver]

It's never gone away. #5937 has recent updates describing current plans. Ultimately we might hope to completely integrate run history into the GUI views, but that's longer term, and it remains to be seen if that would be better than (or preferred over) cylc review style history browsing.

[oliver-sanders]

Sadly, resolving this issue would require a code change (can't just hack the problem away in the environment), this appears to work for me:

diff --git a/lib/cylc/review.py b/lib/cylc/review.py
index 98543dde8..2401a638a 100644
--- a/lib/cylc/review.py
+++ b/lib/cylc/review.py
@@ -596,9 +596,10 @@ class CylcReviewService(object):
                 return cherrypy.lib.static.serve_file(f_name, mime)
             text = open(f_name).read()
         try:
+            text = unicode(text, 'UTF-8')
             if mode in [None, "text"]:
                 text = jinja2.escape(text)
-            lines = [unicode(line) for line in text.splitlines()]
+            lines = text.splitlines()
         except UnicodeDecodeError:
             if path_in_tar:
                 handle.seek(0)

Although this enforces UTF-8, so it likely safer to try this in an except branch, e.g:

try:
    ...
    lines = [unicode(line) for line in text.splitlines()]
    ...
except UnicodeDecodeError:
    try:
        text = unicode(text, 'UTF-8')
        if mode in [None, 'text'];
            text = jinja2.escape(text)
            lines = text.splitlines()
    except UnicodeDecodeError:
        if path_in_tar:
            handle.seek(0)
        ...

To avoid breaking compatibility with any exotic encodings that Cylc Review might currently handle by accident.

[oliver-sanders]

Closed by https://github.com/cylc/cylc-flow/pull/6541