UTF-8 support to elements_scrollable_text_line and elements_string_fit_width

[bolknote]

What's new

• The next commit is for UTF-8 support: UTF-8 added to elements_scrollable_text_line and elements_string_fit_width.
• Fixed bug in elements_string_fit_width (when the width is less than the width of the ...).
• In the elements_scrollable_text_line function bug has also been fixed: the loop now starts with scroll_size - 1.
• There will be another commit in the future with the rewritten elements_text_box.

Verification

• str = furi_string_alloc_set("ЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯЯ");
• elements_string_fit_width(canvas, str, 70);
• printf("%s\n", furi_string_get_cstr(str));
• furi_string_free(str);
• str = furi_string_alloc_set("АБВГДАБВГДабвгдабвгд");
• elements_scrollable_text_line(canvas, 0, 20, 100, str, 3, true);
• furi_string_free(str);

Checklist (For Reviewer)

• [ ] PR has description of feature/bug or link to Confluence/Jira task
• [ ] Description contains actions to verify feature/bugfix
• [ ] I've built this code, uploaded it to the device and verified feature/bugfix

[skotopes]

Please add tests to the example_custom_font app

[bolknote]

I tried to show imagination, I hope this test is suitable. :-)

[skotopes]

Will be merged after https://github.com/flipperdevices/flipperzero-firmware/pull/3322

There are some fundamental issues with u8g2 library that must be addresses first.

[KaliStudio]

have you moved forward with elements_text_box be?

[bolknote]

have you moved forward with elements_text_box be?

Hi! No, I paused this task until merge. Maybe something will conceptually change in u8g2.

[skotopes]

@bolknote @KaliStudio this PR is going to be merged after https://github.com/flipperdevices/flipperzero-firmware/pull/3322 which is planned for next release (0.100)

[KaliStudio]

have you moved forward with elements_text_box be?

Hi! No, I paused this task until merge. Maybe something will conceptually change in u8g2.

@bolknote @skotopes look at the changes I made:

void elements_text_box(
    Canvas* canvas,
    uint8_t x,
    uint8_t y,
    uint8_t width,
    uint8_t height,
    Align horizontal,
    Align vertical,
    const char* text,
    bool strip_to_dots) {
    furi_assert(canvas);

    ElementTextBoxLine line[ELEMENTS_MAX_LINES_NUM];
    bool bold = false;
    bool mono = false;
    bool inverse = false;
    bool inverse_present = false;
    Font current_font = FontSecondary;
    Font prev_font = FontSecondary;
    const CanvasFontParameters* font_params = canvas_get_font_params(canvas, current_font);

    // Fill line parameters
    uint8_t line_leading_min = font_params->leading_min;
    uint8_t line_leading_default = font_params->leading_default;
    uint8_t line_height = font_params->height;
    uint8_t line_descender = font_params->descender;
    uint8_t line_num = 0;
    uint8_t line_width = 0;
    uint8_t line_len = 0;
    uint8_t total_height_min = 0;
    uint8_t total_height_default = 0;
    uint16_t i = 0;
    bool full_text_processed = false;
    uint16_t dots_width = canvas_string_width(canvas, "...");

    canvas_set_font(canvas, FontSecondary);

    // Fill all lines
    line[0].text = text;
    for(i = 0; !full_text_processed; i++) {
        line_len++;
        // Identify line height
        if(prev_font != current_font) {
            font_params = canvas_get_font_params(canvas, current_font);
            line_leading_min = MAX(line_leading_min, font_params->leading_min);
            line_leading_default = MAX(line_leading_default, font_params->leading_default);
            line_height = MAX(line_height, font_params->height);
            line_descender = MAX(line_descender, font_params->descender);
            prev_font = current_font;
        }
        // Set the font
        if(text[i] == '\e' && text[i + 1]) {
            i++;
            line_len++;
            if(text[i] == ELEMENTS_BOLD_MARKER) {
                if(bold) {
                    current_font = FontSecondary;
                } else {
                    current_font = FontPrimary;
                }
                canvas_set_font(canvas, current_font);
                bold = !bold;
            }
            if(text[i] == ELEMENTS_MONO_MARKER) {
                if(mono) {
                    current_font = FontSecondary;
                } else {
                    current_font = FontKeyboard;
                }
                canvas_set_font(canvas, FontKeyboard);
                mono = !mono;
            }
            if(text[i] == ELEMENTS_INVERSED_MARKER) {
                inverse_present = true;
            }
            continue;
        }
        if(text[i] != '\n') {
            line_width += canvas_glyph_width(canvas, text[i]);
        }
        // Process new line
        if(text[i] == '\n' || text[i] == '\0' || line_width > width) {
            if(line_width > width) {
                line_width -= canvas_glyph_width(canvas, text[i--]);
                line_len--;
            }
            if(text[i] == '\0') {
                full_text_processed = true;
            }
            if(inverse_present) {
                line_leading_min += 1;
                line_leading_default += 1;
                inverse_present = false;
            }
            line[line_num].leading_min = line_leading_min;
            line[line_num].leading_default = line_leading_default;
            line[line_num].height = line_height;
            line[line_num].descender = line_descender;
            if(total_height_min + line_leading_min > height) {
                break;
            }
            total_height_min += line_leading_min;
            total_height_default += line_leading_default;
            line[line_num].len = line_len;
            if(horizontal == AlignCenter) {
                line[line_num].x = x + (width - line_width) / 2;
            } else if(horizontal == AlignRight) {
                line[line_num].x = x + (width - line_width);
            } else {
                line[line_num].x = x;
            }
            line[line_num].y = total_height_min;
            line_num++;
            if(text[i + 1]) {
                line[line_num].text = &text[i + 1];
            }

            line_leading_min = font_params->leading_min;
            line_height = font_params->height;
            line_descender = font_params->descender;
            line_width = 0;
            line_len = 0;
        }
    }

    // Set vertical alignment for all lines
    if(total_height_default < height) {
        if(vertical == AlignTop) {
            line[0].y = y + line[0].height;
        } else if(vertical == AlignCenter) {
            line[0].y = y + line[0].height + (height - total_height_default) / 2;
        } else if(vertical == AlignBottom) {
            line[0].y = y + line[0].height + (height - total_height_default);
        }
        if(line_num > 1) {
            for(uint8_t i = 1; i < line_num; i++) {
                line[i].y = line[i - 1].y + line[i - 1].leading_default;
            }
        }
    } else if(line_num > 1) {
        uint8_t free_pixel_num = height - total_height_min;
        uint8_t fill_pixel = 0;
        uint8_t j = 1;
        line[0].y = y + line[0].height;
        while(fill_pixel < free_pixel_num) {
            line[j].y = line[j - 1].y + line[j - 1].leading_min + 1;
            fill_pixel++;
            j = j % (line_num - 1) + 1;
        }
    }

    // Draw line by line
    canvas_set_font(canvas, FontSecondary);
    bold = false;
    mono = false;
    inverse = false;
    for (uint8_t i = 0; i < line_num; i++) {
    size_t j = 0;
    while (j < line[i].len) {
        uint32_t unicode_value = 0;

    // Process UTF-8 multibyte character
    uint8_t first_byte = (uint8_t)line[i].text[j++];
    int additional_bytes = 0;

    // Determine the number of additional bytes
    if ((first_byte & 0xE0) == 0xC0) {
        additional_bytes = 1;
    } else if ((first_byte & 0xF0) == 0xE0) {
        additional_bytes = 2;
    } else if ((first_byte & 0xF8) == 0xF0) {
        additional_bytes = 3;
    }

    // Extract the rest of the bytes
    unicode_value = (first_byte & (0xFF >> (additional_bytes + 1)));
    for (int k = 0; k < additional_bytes; k++) {
        unicode_value = (unicode_value << 6) | (line[i].text[j++] & 0x3F);
    }
            if(unicode_value == ELEMENTS_BOLD_MARKER) {
                if(bold) {
                    current_font = FontSecondary;
                } else {
                    current_font = FontPrimary;
                }
                canvas_set_font(canvas, current_font);
                bold = !bold;
                continue;
            }
            if(unicode_value == ELEMENTS_MONO_MARKER) {
                if(mono) {
                    current_font = FontSecondary;
                } else {
                    current_font = FontKeyboard;
                }
                canvas_set_font(canvas, current_font);
                mono = !mono;
                continue;
            }
            if(unicode_value == ELEMENTS_INVERSED_MARKER) {
                inverse = !inverse;
                continue;
            }
            if(inverse) {
                canvas_draw_box(
                    canvas,
                    line[i].x - 1,
                    line[i].y - line[i].height - 1,
                    canvas_glyph_width(canvas, unicode_value) + 1,
                    line[i].height + line[i].descender + 2);
                canvas_invert_color(canvas);
                canvas_draw_glyph(canvas, line[i].x, line[i].y, unicode_value);
                canvas_invert_color(canvas);
            } else {
            if ((i == line_num - 1) && strip_to_dots) {
                uint8_t next_symbol_width = canvas_glyph_width(canvas, unicode_value);
                if (line[i].x + next_symbol_width + dots_width > x + width) {
                    canvas_draw_str(canvas, line[i].x, line[i].y, "...");
                    break;
                }
            }
            canvas_draw_glyph(canvas, line[i].x, line[i].y, unicode_value);
        }
        line[i].x += canvas_glyph_width(canvas, unicode_value);
        }
    }
    canvas_set_font(canvas, FontSecondary);
}

[skotopes]

@KaliStudio your code doesn't make sense: text_box uses predefined system fonts and they don't have any unicode characters, so it's going to render nothing.

[skotopes]

@bolknote I think couple things must be done for this to be merged:

• FuriString unicode support must be improved first: add proper unicode iteration and etc ...
• Methods that want to implement unicode must use FuriString
• Dangerous pointer arithmetic must be avoided
• There should be one and only one unicode codec (can be a part of FuriString)

[bolknote]

Hi! Yes, I was thinking about it, but I wanted to move step by step, all improving the Unicode code support with my commits.

Besides, I'm not sure how best to do it. There are two options.

1. create a second set of functions with utf8 prefix (like furi_string_utf8_length)
2. make a flag inside the FuriString structure that the string is in UTF8 and call functions depending on it.

Which way is preferable?

[skotopes]

@bolknote easiest option will be to add second set of methods for unicode processing only.

[KaliStudio]

@KaliStudio your code doesn't make sense: text_box uses predefined system fonts and they don't have any unicode characters, so it's going to render nothing.

For my part, characters like é transform into ã©. Hence this modification

[bolknote]

Hi!

I want to close this task, because I need to write functions to work with Unicode and only then modify the "elements" functions.

In the coming days I will open another PR with features to work with Unicode. After that, after some time, there will be a PR with modified "elements".