lisp-binary icon indicating copy to clipboard operation
lisp-binary copied to clipboard

PAD-FIXED-LENGTH-STRING is wrong for non-8-bit string encodings

Open heegaiximephoomeeghahyaiseekh opened this issue 8 years ago • 0 comments

lisp-binary::pad-fixed-length-string adds its padding before encoding. This works fine for encodings such as ASCII and Latin-1, but will fail for encodings such as UTF-8, in which the byte length may differ from the character length.

Padding the string after encoding might not work either, since the padding bytes would have to respect, (and therefore the writer generator must be aware of), the rules of each encoding.

One way to fix it could be to use a binary-search algorithm to find the right number of padding characters to add to the pre-encoded string to get the desired post-encoding length. In the worst case scenario, this could require re-encoding the same string dozens of times.