Hot reloading of layout

[PvdBerg1998]

Like discussed on Discord - perhaps it's nice to provide some sort of "quick iteration" possibility, for example using Lua.

[krooq]

I think this is an essential feature for designing any kind of complex UI. It takes me 30 seconds to recompile my 90 line main file when I make a small change to the view which is just far far too long for iterative development.

EDIT: I have done a lot of work with Flutter and it's hot reload is amazing. I think nothing that extensive is needed but just reloading the view on the fly would be great.

[clawoflight]

It might be worth thinking about clever imgui integration instead, at least in the short run?

[ethanpailes]

https://github.com/hecrj/iced/issues/303

[tangmi]

I'm taking a stab at this and I'll upload what I have once I get a bit more working, but I'll try and enumerate what I've learned so far.

I think this requires two main components:

1. Markup/templating language: Something like XAML, JSX, Handlebars + HTML, etc. I think this is a very useful component for iced to have, regardless of hot swap capabilities.
   • The template engine needs to support some control flow (if, for each, etc) based on user state.
   • The runtime component of the markup language needs to:
     • Resolve custom widgets into iced::Elements.
       • This might need widgets to have some declarative metadata describing attributes and events, which would allow custom and "builtin" widgets to be treated the same
     • Store/reference widget state. (Note: This could happen automatically! It'd be an awesome gain for iced user producivity!)
     • Reference message construction functions for callbacks.
       • This probably needs reflection (codegen or runtime).
         • I'm currently playing around with a HashMap<String, &dyn Any> that the app fills on initialization and downcasting to the exact function pointer type, but am running into an issue with Any::downcast to a function pointer.
         • Also going to look into leveraging serde's codegen to generate messages.
   • This could probably be built as a library, given changes in iced for support.
2. Hot reload runtime component: This will actually do the hotswapping of the resolved markup and ends up being pretty trivial (file watcher with an optional code recompiler) once we have the markup language.
   • This can probably be done as a library, with minimal or no support needed in iced itself.
   • I can imagine a setup where the layout itself is reloaded from the markup language (almost) instantly and when state/callbacks change the entire app could be recompiled and the binary hotswapped in, preserving app state.

Some additional notes:

• My current prototype uses ron as the markup "language" and can currently generate/reload non-interactive stateless views. Most features I described above are not implemented.
• It'd be nice to eventually include the markup language and reloading as "blessed" crates.
• I'm currently ignoring the issue of how to make this zero/low overhead in release builds.

@hecrj What are your thoughts on an iced layout markup language? I'll happily file an issue with all the details/prototype I have if you're into it.

[tangmi]

https://github.com/tangmi/iced-hotswap-prototype

Here's a working proof-of-concept of the following features:

• Ron-based markup language templated with handlebars
  • currently supports what's in iml::Element: Column, Row, Text, Button, Slider (with minimal parameters)
  • resolves templates with user-provided state
• automatically creates, owns, and maintains widget "ephemeral" state (i.e. button::State)
• resolves callbacks with user-provided "message" struct
• can read values from user-provided state

Some caveats:

• I'm heavily leveraging serde's code generation to e.g. discover the variants of a user-provided messages enum.
• A lot of features are missing
• no errors are handled gracefully (if you spell your markup file wrong, it'll immediately crash).
• I'm abusing std::mem::transmute in a couple places to avoid "solving" where to put lifetime annotations
• ~After saving the file, one needs to interact with the app (mouseover) to get it to reload. This could be fixed by implementing a Subscription that watches the file system~ I've added a Subscription that just wakes up the app every half second to force a rerender. (there's definitely a smarter way to do this).

[hecrj]

@tangmi Hey! That's really cool!

I really like how it keeps update code type-safe. I imagine we could build our own Application trait wrapping iced::Application to hide the Hotswappo struct and view and provide the state directly to update. This should also help us use a different strategy for release mode (ideally generating Rust view code from the template and using the iced::Application directly for no runtime cost).

I would also probably rename the various callback_name to the actual widget method names (i.e. on_press for a Button, etc.), but I imagine the details of the language are not very important right now.

Overall this is very, very promising! It could ease development process a lot and help us build view editors!

[hecrj]

automatically creates, owns, and maintains widget "ephemeral" state (i.e. button::State)

It will be very tricky to do this well without an important performance impact and logic errors (i.e. diffing).

At least for the time being, I would require the user to define widget state in the application state and provide it in the markup. Then, if the user adds a new widget in the template and the state field doesn't exist, we can fall back to the current naive tracker for hot reloading. However, I would make it a requirement for release builds (i.e. compilation error if the state fields do not exist).

[tangmi]

Glad you like it! Thanks for the feedback!

I like the idea of hiding Hotswapo behind a separate Application trait opening up the possibility for transparent hotswapping in debug builds for the user.

For the markup side, I was imagining two "runtimes":

• Dynamic: one with a similar lineage to the prototype--intended to load dynamically and have additional hot-swapping abilities at the cost of performance depredations.
  • This is the one hotswapping would use.
• Static: a compile-time affair that generates rust code for the view (that includes the branching as rust control flow) and a struct that contains the ephemeral state (please correct me if there's a better name for this) of exactly the widgets that appear in the markup file.
  • In this case, I don't think the user needs to provide widget state, which is an otherwise very mechanical action (every widget one adds needs its associated state, it's very unlikely that two separate widgets want to share the same state, etc).
  • This is the one non-hotswapping (release builds?) would use.

In both cases, the result would be a function something like fn view(&loaded_layout, &mut user_state) -> Element<'_, M>. I might be handwaving the difficulty of implementation/maintaining cost this dual-action runtime, but I believe some combo dynamic/static layout file solution gets us both perf and flexibility.

[aevyrie]

@tangmi I was about to try something similar, nice work! For my needs, I was thinking of a different, possibly simpler, approach. I'll expand on that here in the hopes that it adds to this discussion.

Instead of generating any rust layout code, my priority lies in hot-reloading the styling of existing widgets. The plan was to define all styles and sizes in a single serializable object, I'll call StyleSheet for now. I would then reference this StyleSheet whenever I define a widget in an iced fn view(). Similar to CSS classes.

The hot swapping logic would live in an async loop constantly looking for edits to a file, then triggering an iced Command when an edit is detected. This Command would use serde to import this (TOML?) markup file containing the serialized StyleSheet object. This would allow me to rapidly iterate on the styling (widths, spacing, coloring, borders, etc) of my framed out UI. Once I'm happy with it, I can just drop this tweaked StyleSheet object into my rust code to remove the need for the hot-reloading runtime.

This approach obviously doesn't allow for generating view code on the fly, but I haven't found that to be an issue so far. Compartmentalizing groupings of reusable widgets into "components" that I can then drop into the top level UI has been painless, and makes the top-level code very readable when the entire gui isn't expressed in a giant nest of .push()s. I've also found that iced's Elmish way of defining layout makes it dead simple to get the layout, if not the styling, where I want it to be very rapidly.

I'm also skeptical of the utility of generated view code, because it quickly becomes customized - especially views that hold lists of things - with .filter()s and .fold()s. Once I've prototyped a gui and hook up all this logic, the next time I want to tweak my view code via hot-reloading, I'd have to manually reconfigure all this logic, which seems to defeat the purpose!

In my mind, that's like generating HTML, when what I really want is to just hot-reload my CSS.

With all that said, maybe what I have in mind isn't the right approach, or representative of most users' needs! I think this is really important work for iced and I'm glad you're tackling it. 🙂

[tangmi]

@hecrj I'll keep poking at this when I can find time. I think the main takeaway from my investigation is that both a markup language and hotswapping can probably be developed as a library with no* changes in iced needed (great work on library design!).

*I think some mechanically consistent declaration of widgets and their state/attributes (not sure what this looks like!) can help simplify the markup language implementation and allow for custom widgets.

[hecrj]

I think some mechanically consistent declaration of widgets and their state/attributes (not sure what this looks like!) can help simplify the markup language implementation and allow for custom widgets.

Could you elaborate? What do you mean by this?

[tangmi]

(Be aware this is a half-baked idea)

I think I mean some consistent mapping from markup elements to code. E.g. If all widgets provide an "attribute setter" interface (e.g. Widget::set_attribute(attribute_name: &str, attribute_value: _)), then it's pretty feasible to go from

Button(
    content: Text("button"),
    on_click: "ButtonClicked",
    disabled: false, // <- I know this isn't how `Button` handles disabled currently!
),

to

let markup_element = todo!("deserialized from file");

let mut button = Button::with_state(widgets_state.get(markup_element.get_id()));
button.set_attribute("content", get_element(markup_element.get("content"));
button.set_attribute("on_click", get_callback(markup_element.get("on_click")));
button.set_attribute("disabled", markup_element.get("disabled"));

without any special understanding of Button. We just need to know that widgets have attributes that can be set (compared to my prototype, which needs to know about every widget, which is an unbounded set if we include user implementations of iced_native::widget::Widget).

...although I'm unsure how to make this typesafe! The widget might need to have some manifest declaring its attributes and the types of its attributes? Maybe through some code generation (this would require changes in iced)?

[aevyrie]

I think I mean some consistent mapping from markup elements to code. E.g. If all widgets provide an "attribute setter" interface

Could you implement a trait for each widget in the meantime, e.g.

trait Attribute {
     fn set_attribute<T>(&mut self, attribute: String, value: T);
}

then leave it up to the user to implement this trait if they want to use a custom widget with hot reloading?

[tangmi]

I think the attribute would need to take &dyn Any (+ Clone?), otherwise, I'm not sure what an implementation could look like

struct Foo {
    val: i32,
}

impl Attribute for Foo {
    fn set_attribute<T>(&mut self, attribute: String, value: T) {
        if attribute == "val" {
            self.val = value; // self.val expects `i32`, value is `T`. !!
        }
    }
}

[aevyrie]

Something along these lines?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a2bca4e72f661bb39783c1c5045b137a

[tangmi]

I think something like that world work! To support non-num-primitives, I think it'd still need Any... we're getting into weird territory, but maybe something like this?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=082e30cb889efedf3b83d0b4e9471f51

Edit: this assumes we know the type of each attribute upfront... I'm not confident this is knowledge the markup engine would have without some widget manifest...

[aevyrie]

I've made some progress on (my version of) this. I ended up going the route of directly placing variables in the attribute setter calls, e.g.:

widget_xyz.spacing(stylesheet.spacing_outer)

The stylesheet is just a struct that contains these appearance variables, as well as a separate struct of colors, so the appearances can reference them. I'm still working on the file watching and serde side of things now, but it looks like the fundamentals are proved out.

Limitations

This implementation does not allow dynamic renaming of appearance names, because these are set at compile time. So, once you have set widget_xyz.spacing() to use stylesheet.spacing_outer, you can't change it at runtime; all you can change is the value of spacing_outer.

Benefits

The immediate benefit of this is type checking. The compiler can guarantee that the type of spacing_outer is compatible with .spacing()'s signature. An alternate implementation might instead use a str to look up a matching appearance from a HashMap. The risk with that is you can assign a nonexistent appearance name (typo, stylesheet change), and it won't be caught at compile time. It's possible this could be solved with a macro, or maybe forcing a panic if an appearance call returns None while hot reloading is disabled.

[aevyrie]

Well, I have a proof of concept working! The iced UI updates immediately when saving the stylesheet. (Note the red text)

Usage in this case is: .color(stylesheet.color(&stylesheet.text_color_h1))

I can change the color in this example by either changing the rgb values of the named "text" color, or by changing text_color_h1 to a completely different named color such as "primary". I chose this example because it relies on an extra layer of abstraction: colors can be added and referred to by name dynamically. Here's an example of adding a new named color and changing the reference to it:

I'm looking forward to being able to tweak the UI without recompiling. 😄

[shujaatak]

@aevyrie If you don't mind, would you please provide some implementation details or show us your code?

[aevyrie]

@shujaatak Of course. Please excuse the state of the code, it's very much in flux. https://github.com/aevyrie/tolstack/blob/master/src/ui/style.rs

I'm not super happy with the verbosity required for a stylesheet - there is the Rust boilerplate of defining the new field in the stylesheet struct, then instantiating a value in the instance of the struct. As far as how the system works, there are NamedXYZ structs for each type of styled object, like a button or container, that implement the Named trait. This trait takes in the string of the style class you've defined in your stylesheet (e.e a named color such as "primary"), and the list to use as a style lookup. The string name is then resolved to generate an Iced style using the lookup, and panics if the style class name provided is invalid.

This is really just a prototype that evolved as I started using it to build out the application. I'm sure this concept could be fleshed out in a more robust way if there is interest in doing so.

[pickfire]

https://fasterthanli.me/articles/so-you-want-to-live-reload-rust maybe related? We could even hot reload the whole application.

End result https://www.youtube.com/watch?v=o1iqV5k6-QU

[ethanpailes]

@pickfire I think a dynamic library approach would still run into the problem that there is a compile step in the middle of your style/layout tweaking loop. It seems likely to be too slow.

[pickfire]

But compiler could guarantee there is no issue with the data type, say if we use json there is no validation or whatsoever, at least when it compiles it usually works. IIRC there is something to speed this up, I think one can change value in rust code and reflected directly, I saw a crate for this but I forgot which is it.

[mobile-bungalow]

@pickfire You are probably thinking of the dymod crate. I've tried the dymod method (expirement here), it has the following issues.

1. View logic must be in a separate crate. This becomes a problem if you are using more complex global state, or if you appstate contains many types from heavy dependencies.
2. You need a concrete Renderer implementation in the dymod. this means you already have to relink wgpu and its dependencies, on my macbook that takes 9 seconds.

solutions to both these problems are burdensome. problem 1 might require you to turn your appstate into a heavily encapsulated trait object to avoid exposing dependencies to the dymod. Problem 2 could be solved a number of ways, but not without reorganizing Iced's trait structure.

reasons to strongly prefer the dymod:

1. no need for scripting or markdown. You will never have to migrate work in markdown to rust. you will never need to ship markdown files with your application. You won't have to wait for the markdown crate you use to catch up to Iced's latest release cycle, or migrate if it gets abandoned.
2. there will be no parallel effort to track all of iced's widget in an enum, which would exclude any third party widgets!
3. static typing. sweet sweet static typing.
4. only need to iterate on one element? return only that element from the dymod and keep the rest of your code consistent.

It seems like you should use the dymod method if you want to avoid registering widgets in some kind of interpreter beforehand like Qml.

[kkharji]

Oh hot reloading is so needed. Hope it becomes a reality for iced.

[90-008]

What about making use of something like https://github.com/rhaiscript/rhai? It seems that it can be integrated with iced quite easily, even better is it allows syntax extensions so in the future it could even be possible to integrate iced specific syntax. Considering making a prototype that integrates with iced.

[90-008]

Alright, so I played with the idea a little bit and I think it should be possible to integrate iced with Rhai, but it will require changes in iced itself.

Problem 1. Rhai requires that types that will be registered implement Clone, which almost all of iced's widgets don't. Most importantly, Element doesn't. Problem 2. Potential lifetime issues (?), but I didn't dig into that much (couldn't use any widgets anyways, problem 1). (This seems like it can be "just" fixed by making use of 'static lifetimes)

~~Problem 1 could be fixed (?) by using Rc<RefCell<>> wrappers. But this has the problem that we have to redefine each and every method again, to take Rc<RefCell<>> wrapped version of the widget instead. This could be done with a proc macro, but it's still effort.~~ It looks like this can only really be solved by making Element and other widgets cloneable.

I pushed a little "demo" with Text hot reloading to https://github.com/yusdacra/iced_rhai, and what follows is a video of it.

https://user-images.githubusercontent.com/19897088/121803630-3ba1b900-cc4b-11eb-8c82-d64c2394e5dc.mp4

[90-008]

I was able to (hackily) implement Rc wrappers. I have pushed it to the repository, below is a sample with Row, Column and Container.

https://user-images.githubusercontent.com/19897088/121820141-cf01db00-cc99-11eb-8b06-82bcb2748965.mp4

Another way this could be implemented without unsafe code is, we would practically "copy" all of the widgets' "data" part, create methods, derive Clone etc. This would probably take more time (?) to do though.

We could also implement Clone for the iced_native Widget trait. I tried doing this with https://lib.rs/crates/dyn-clonable, it complained about Renderer and Message generics not being Clone. Even if we could get Message to require Clone, Renderer is too much for that.

[johannesvollmer]

Avoiding markup is one of the core goals of elm. Both Elm and Rust are less verbose than XML. It is in fact one of the selling arguments of elm.

If your only concern is the compile times, have a look at bevy. I think they use dynamic linking (dll) for development, which results in insanely fast compile times, and only use static linking for release builds. Here's a link, scroll down a few paragraphs. Maybe this could be explored with iced too, regardless of hot-reloading? This way, only iced would be pre-compiled once, and only the user code is compiled in every iteration.

Using markup/templates removes the dynamic behavior. Approximating dynamic behavior using yet another alien syntax for loops and branching sounds like a lot of effort without much value to me. Furthermore, using a template language adds a completely new barrier. And then it has to be thrown away once you realize that you need actual dynamic behavior, and you'll need to reimplement it in Rust.

Using rhai or some other real dynamic language seems much more reasonable to me. However, at that point, it would probably never be favorable to actually use iced with Rust directly. You'd most probably write everything in rhai instead of dealing with transferring objects from one language to another, which further can introduce subtle bugs. Is that the route you want iced to go? If not, rhaiced should perhaps be an independent project.

I personally choose to use Rust because of the powerful type system, the package manager, and the performance. At least two of these three selling points are lost when you use a dynamic scripting language. If all you desire is fast compile times, supporting dynamic linking for Rust development might be the way to go. Otherwise, once the user realizes that rhai performance (although neat) can not achieve compiled speed, they can't simply recompile with a different flag, but they will have to rewrite core logic in Rust. And deal with transferring objects across the language border.

If Rust is still too verbose compared to rhai, perhaps we should improve the Rust API instead of using a completely different language. I'm very pleased with the current iced design, but with enough love we could probably make it a little more concise.

My point is: let's not forget that there's a lot we can improve in Rust. Conciseness and compile speed are issues that can be solved that way. We don't have to add an opinionated secondary language to tackle these issues. I assume most of use actually like Rust, we choose it for a reason, right?

[Cambloid]

some sort of markup language for iced would be awesome. to keep the crates lightweight this could be in a sparate crate like: iced_markup. like iced_audio for audio related widgets.