sfm-ui

sfm-ui copied to clipboard

Confirmed in SFM sandbox that this is a problem with the current release, not just the twitter-v2 branch.

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 624, in _receive_callback return on_m(message) if on_m else self.receive(decoded, message) File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 590, in receive [callback(body, message) for callback in callbacks] File "/usr/local/lib/python3.8/site-packages/kombu/messaging.py", line 590, in <listcomp> [callback(body, message) for callback in callbacks] File "/usr/local/lib/python3.8/site-packages/sfmutils/consumer.py", line 111, in _callback self.on_message() File "/usr/local/lib/python3.8/site-packages/sfmutils/exporter.py", line 125, in on_message export_formats[export_format][1](table, filepath) File "/usr/local/lib/python3.8/site-packages/sfmutils/exporter.py", line 379, in to_lineoriented_json f.write(chunk) TypeError: a bytes-like object is required, not 'str'

Based on my testing, I think this is the fix for exporter.py in sfm-utils:

def to_lineoriented_json(table, source):
    """
    Function to enabling PETL support for exporting line-oriented JSON.
    """
    source = write_source_from_arg(source)
    encoder = DateEncoder()
    with source.open("w") as f:
        for d in _dicts(table):
            f.write(encoder.encode(d))
            f.write("\n")

I'm not sure why this function was writing files in binary previously -- maybe there was some performance advantage? But it no longer seems to work...perhaps a Python version issue.

See PR 48 on sfm-utils.