dumbo
dumbo copied to clipboard
Reading text as typedbytes affects lines with encoding other than utf8
Streaming backend assumes that input format is typedbytes even if -inputformat argument is 'text': https://github.com/klbostee/dumbo/blob/release-0.21.36/dumbo/backends/streaming.py#L81
This leads to apply typedbytes.PairedInput to all input lines: https://github.com/klbostee/dumbo/blob/release-0.21.36/dumbo/core.py#L380
Appling util.loadtext instead of typedbytes.PairedInput resolves this issue.