Tracking issue: Features for egui

[valadaptive]

Similar to the Blitz tracking issue, this is what egui will eventually need from Parley.

Parley

• [x] Text wrap styling (https://github.com/linebender/parley/pull/315)
• [x] Text truncation with ellipsis
• [ ] serde support for some types (the selection ones at least)
• [ ] Vertical alignment options, especially for InlineBox (https://github.com/linebender/parley/issues/291)
• [ ] Ability to set line.offset? (necessary for egui's LayoutSection::leading_space)
  • [ ] This can't just be a visual thing because of hit testing; Layout needs to agree on where everything is
• [x] Absolute line height
• [x] Custom family names (https://github.com/linebender/parley/issues/117)
• [x] Inline box fix (https://github.com/linebender/parley/pull/299)
• [x] Don't round vertical metrics (https://github.com/linebender/parley/pull/297)
• [x] RTL jank (https://github.com/linebender/parley/issues/298)
• [ ] Support the tab character (https://github.com/linebender/parley/issues/302)
• [ ] AccessKit API improvements (https://github.com/linebender/parley/issues/310)
• [ ] Technically Swash: tighter glyph bounds (https://github.com/dfrg/zeno/pull/15) (merged but not released)
• [ ] A newer/better "attributed text" API. Currently I'm using TreeBuilder since I need to push entire style sets at once, but TreeBuilder allocates a new String every time, which is unnecessary.

Fontique

• [ ] SystemUi doesn't properly fallback in some cases (e.g. Arabic text, macOS shortcut symbols) on my machine; SansSerif does (https://github.com/linebender/parley/issues/323)
• [ ] Size adjustment multiplier for custom fonts (https://github.com/linebender/parley/issues/324)

API rework?

As discussed in https://github.com/linebender/parley/issues/325, a lot of this would be way easier to implement if Parley exposed lower-level APIs that let us position each line ourselves. In particular, egui needs to support flowing text around previous text, which I'm currently hacking together with InlineBox and shifting things around vertically.

[nicoburns]

What do you mean by "In particular, egui needs to support flowing text around previous text"? I don't suppose you have a screenshot example?

[valadaptive]

What do you mean by "In particular, egui needs to support flowing text around previous text"? I don't suppose you have a screenshot example?

As discussed in https://github.com/linebender/parley/issues/325, egui has a Label widget. Because it's immediate-mode, layout works one widget at a time. So if we add a Label widget and then put another Label widget afterwards, the second Label widget needs to treat the first Label as an inline box or floated box.

Because, again, the UI and layout is immediate-mode, we don't know ahead-of-time where all the inline boxes will be, and can't just stitch together every bit of text in a left-to-right layout. When we place a widget into a layout, it calculates its own size and then advances the layout cursor by that amount.

Even if we did have the luxury of two-pass layout, it would punch through a lot of layers of abstraction to have to convert every inline egui layout into a corresponding Parley layout, where every non-text widget becomes an InlineBox and Parley dictates its alignment (I'm not sure how xilem/masonry deals with this either). Parley is a text layout library, not an HTML inline flow layout library or a widget layout library. As a result, I believe InlineBox isn't a good way to implement interspersing text with widgets.

The API you put forward in #325 and #99 seems pretty reasonable, but I think we can simplify it a bit. If we are willing to forgo doing full vertical CSS-style layout and alignment in Parley itself, or even just implementing it in a "Parley extras" module, I think we can replace the "inline box" and "floated inline box" APIs with an API that lets you specify interruptions in the flow of text, which will stop break_next early when reached. If the API consumer is doing layout, then it's the only party that needs to know about the interloping element's dimensions and alignment.

[emilk]

To add to that explanation with something visual:

[some previous widget taking up a known width] New label text
that should wrap at a certain width, and of course not paint
over the previous widget

Currently egui handles the existing widget by handing a "first-line-indent" to the old egui text layout engine (FWIW).

[nicoburns]

If the API consumer is doing layout, then it's the only party that needs to know about the interloping element's dimensions and alignment.

Hmm... if the API consumer is doing layout, then is any of the line breaking code (greedy.rs) useful? Would you basically just want an iterator over shaped glyphs, sizes, and advances and then you'd do line-breaking yourself?

Parley is a text layout library, not an HTML inline flow layout library or a widget layout library.

Well, it's kind of both. For me having that in Parley (or some generically usable library that isn't coupled to Blitz) is very much a goal, because "web-compatible layout as a library" would (IMO) be fantastically useful for all sorts of things. And I guess I'm a little sad about the idea of moving too much of layout out of Parley as whatever is in Parley can be shared and reused. Whereas the stuff that is coupled to GUI framework is destined to be reimplemented by each one (albeit some things may end up being done differently by different frameworks).

Parley dictates its alignment (I'm not sure how xilem/masonry deals with this either)

I don't Xilem/Masonry does deal with this atm (I don't think it supports widgets/boxes in line with text at all yet). My plan for Blitz for this was to add a first_baseline: f32 field to inline boxes which would be an offset from the top of the box which baseline alignment would use as the baseline. That (along with all of the vertical-align options (including a pixel offset)) would be sufficient for css-style vertical alignment. I'm not what other options you might want in egui.

As discussed in https://github.com/linebender/parley/issues/325, egui has a Label widget. Because it's immediate-mode, layout works one widget at a time. So if we add a Label widget and then put another Label widget afterwards, the second Label widget needs to treat the first Label as an inline box or floated box.

Couldn't egui retain the Parley layout from the first Label (within a single frame) and keep appending to it until the end of the inline layout is reached? I guess if you need the layout of the first Label before you move on to the second one then that wouldn't work today, but we could add "incrementally append to layout" functionality. Basically, interleaving control flow between Parley and Egui's layout rather than doing two passes.

it would punch through a lot of layers of abstraction to have to convert every inline egui layout into a corresponding Parley layout

I'm very much sympathetic to this. I don't fully understand how immediate mode layout/rendering works. But I can imagine there might be restrictions there. And I definitely think that Parley's API being awkward for your use case is sufficient reason to create a better suited API (even if the existing API could technically be made to work).

I think we can replace the "inline box" and "floated inline box" APIs with an API that lets you specify interruptions in the flow of text, which will stop break_next early when reached

Totally agree with this. I'm still going to want "full CSS-style layout and alignment", but we could totally build that on top of a generic "interruption" abstraction.

[valadaptive]

Hmm... if the API consumer is doing layout, then is any of the line breaking code (greedy.rs) useful? Would you basically just want an iterator over shaped glyphs, sizes, and advances and then you'd do line-breaking yourself?

That code is the sort of thing I'd like to disentangle into separate text-handling and layout-handling components. It handles a bunch of complicated text analysis like whitespace handling, ligatures, bidirectional text, and other things that the API consumer probably wouldn't think to handle unless we did it for them. But it also handles line spacing and inline box alignment, which could probably be split out into a completely different module.

I don't actually think line breaking is part of layout. Line positioning is, and it's often tied to line breaking, but there's no reason that has to be the case. "Line break" doesn't even mean you place the next bit of text below the previous one! For multi-column layouts, or flowing text around images, the next run of text could be above or to the right of the previous one:

Floating images in the middle of text is actually a really good example of something that the CSS model can't do, but that Parley should be able to support--it's an extremely common feature in word processors.

Couldn't egui retain the Parley layout from the first Label (within a single frame) and keep appending to it until the end of the inline layout is reached? I guess if you need the layout of the first Label before you move on to the second one then that wouldn't work today, but we could add "incrementally append to layout" functionality. Basically, interleaving control flow between Parley and Egui's layout rather than doing two passes.

This is exactly the kind of abstraction-breaking I'd prefer to avoid. Let's say we change Parley so that we can incrementally add to a Parley layout. We then create a new Parley layout for every egui child UI, and call into it to ask it where we should position the next element. Well, now it seems like Parley isn't a text layout library but instead a UI layout library like taffy! If you want to use Parley to get selectable text with proper shaping and font fallback, now it's made its way into your public layout API.

Well, it's kind of both. For me having that in Parley (or some generically usable library that isn't coupled to Blitz) is very much a goal, because "web-compatible layout as a library" would (IMO) be fantastically useful for all sorts of things. And I guess I'm a little sad about the idea of moving too much of layout out of Parley as whatever is in Parley can be shared and reused. Whereas the stuff that is coupled to GUI framework is destined to be reimplemented by each one (albeit some things may end up being done differently by different frameworks).

That last point, about layout being reimplemented by each GUI framework, is a feature and not a bug to me. Layout is one of the main things a GUI framework is meant to handle! There was even some discussion on Zulip recently about seeking better layout solutions than the CSS box model. I think it's completely backwards for a GUI framework's layout API to be dictated by the text rendering library it uses, of all things.

There are also a couple of other concerns: first, if we want to implement CSS layout in Parley, we need to implement all of it in Parley. That means not only the existing CSS inline layout spec, but also keeping up with any new additions. If Blitz wanted to support a CSS feature that just got added, for instance, you'd have to implement it in Parley and wait for it to land here first. Second, when I say "all of it", that might include more than just the inline layout spec that you might be thinking of. The inline layout model makes reference to the CSS2 visual formatting model several times, especially when it comes to floats.

I would be fine implementing CSS inline layout as another library atop Parley (maybe as another crate in this repo, like fontique is), but I don't think there's any need to couple that functionality to Parley.