Dictu icon indicating copy to clipboard operation
Dictu copied to clipboard

wip: utf-8 support in strings

Open liz3 opened this issue 1 year ago • 1 comments

Add Unicode support to strings

Resolves: https://github.com/dictu-lang/Dictu/issues/317

What's Changed:

This updates the implementation of the string API to support unicode using the utf8.h library.

Theres still a few incomplete things, some fuctions do not need unicode versions(strip functions). But on the other side there are things where the library does not provide full functionality which is needed for isUpper/isLower for example which do not wrong, forcing the usage of the c std lib apis.

Should String.len() return byte length or character length, what if the string has invalid utf8? calculating the length upfront for large strings, always might be expensive.

And not enough tests are added.

Type of Change:

  • [ ] Bug fix
  • [X] New feature
  • [X] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Housekeeping:

  • [ ] Tests have been updated to reflect the changes done within this PR (if applicable).
  • [ ] Documentation has been updated to reflect the changes done within this PR (if applicable).

Screenshots (If Applicable):

liz3 avatar Sep 18 '24 01:09 liz3