Dev tools abstraction

[alice-i-cecile]

Dev tools are an essential but poorly-defined part of the game development experience. Designed to allow developers (but generally not end users) to quickly inspect and manipulate the game world, they accelerate debugging and testing by allowing common operations to be done at runtime, without having to write and then delete code. To support a robust ecosystem of dev tools, bevy_dev_tools comes with two core traits: ModalDevTool (for tools that can be toggled) and DevCommand (for operations with an immediate effect on the game world). These are registered in the DevToolsRegistry via methods on App, allowing tool boxes (such as a dev console) to easily identify the available options and interact with them without the need for ad-hoc user glue code.

Co-authored with @matiqo15 and @rewin123.

Read it RENDERED here.

[makspll]

This is really cool! And since I am maintaining bevy-console I am very excited about the possibilities here! A few things though:

First of all why have parse_from_str on modals at all? For commands this is clear, a command has a one-to-one relationship with some string representation, for a modal this is less clear cut, and a parse_from_str feels more like deserialisation. If anything, instead of returning a Self instance, modals could have a react_to_str method but this would step on DevCommand territory.

Also parse_from_str seems to be very un-opinionated, which may be on purpose, but I can see different standards arising, i.e. do we include the dev tool "executable" name as first argument, string separators, how do we interpret quoted strings (i.e. does "arg" == arg).

I don't think we necessarily should depend on clap directly, but we could have some more abstractions here, I am thinking something along the lines of a smaller https://docs.rs/clap/latest/clap/trait.FromArgMatches.html, so a slight abstraction from text, which allows room for different standards. For example as an extreme example: both "{ my_arg: 123 }" and "--my-arg 123" would come down to the same thing via parsers, and we could even provide a default one (clap?) and hell could later store that with the dev tools.

let matches = parse(input);
let val: usize = matches.get<usize>("my_arg"); // 123
// or
let val: String = matches.get("my_arg"); //123

EDIT: I suppose not forcing some sort of "arg" based construction method for modals would then rely on authors supplying commands to modify all config properties of the modal. In which case I can see why having such a method would be beneficial, however if the modal has 20 properties, your console might as well support multi line json haha :P

[alice-i-cecile]

First of all why have parse_from_str on modals at all

I've expanded the toolkit example to try and clarify this: you need it to be able to allow for free-form runtime config of these resources. Obviously you could make dev commands for each way you might want to configure the resource, but I was thinking it would be more user-friendly to allow them to just set the struct's values directly.

Also parse_from_str seems to be very un-opinionted, which may be on purpose, but I can see different standards arising

Yeah, I'm not sure of the best way to handle this. I'd like to interoperate with std's FromStr trait, but it really doesn't impose any structure at all.

however if the modal has 20 properties, your console might as well support multi line json haha :P

Yeah, I actually really want to support parsing these objects from the new bsn format at some point!

[rewin123]

The RFC looks really good! I really like the good analysis done on usage and what should be classified as dev tools and will be used and what will not. From this analysis comes for me a reasonable and good trait structure.

However, I think the RFC did not address one important issue in my opinion.

Part of the dev tools may want to display debugging information on the screen or get some input from the developer. For example, the fps overlay displays the number of fps in the bottom right corner, and the entity inspector may want to show some editable fields and buttons near the right edge of the screen. Or the realtime statistics render graph will want to display rendering times information somewhere in the static area. And if multiple dev_tools claim the same area of the screen to display, it will be a mess. And obviously dev_tools can and will conflict over displaying information. Accordingly, the question not solved and not asked in my opinion in RFC is "how is it planned to organise the location of the displayed information of dev modules?". (Or how do we start building editor UI from dev tools xDDDD, sorry).

[alice-i-cecile]

@rewin123 great question! I've added more thoughts on this to both "Rationale" and "Future work" justifying why it's not included currently: https://github.com/bevyengine/rfcs/pull/77/commits/45506d2e923e6ce261790e2f1c69476b6c158c87

[codercommand]

On the topic of commands, personally I've come to really like menus like F1 on VScode and ctrl+k on Github.

Personally I like the idea of commands being a feature that has ui inspired from Githubs ctrl+k menu

Then as for features it works like traditional console and have take commands, but it can also provide context. This allows it to provide context relevant info like the file explorer right click context menu or you can search through sub commands/directories.

I'm not sure on the best way it should work. Since it's a hybrid between a console and a context menu, maybe have the user right click and if they have something selected in the scene it can be used by the command.

Imagine your workflow looking something like click on thing, right click to get menu, type command or optionally click on a proposed option. And if you click on an option, it could open more suboptions, etc.

[alice-i-cecile]

Great suggestion: I love command palettes too, and I've added them as an interesting example of a potential toolbox to consider when designing.

[codercommand]

Something else to propose. I've noticed some engines like Godot and Defold don't support multi monitor support.

Since this system will need to define what takes screen space and bevy already comes with multi window support. Maybe make it that there is a tools window and game window?

This would allow for x2 monitor workflows.

You'd have grid slots on the game/main window that can be enabled and disabled, and you can choose what displays where the same as nividias hud layout thing.

Then the second window would be used by other tools that require more screen space. These ones would take the entire window except for a strip at the top which would be a tab browser. This would allow for full window tools, and you can click between their tabs to swap between them like in a browser.

With this setup you now have dual monitor support.

If you wanted single monitor support, you can make the game window a tab. Now it shares the same window as the other tools.

[codercommand]

I can imagine a plugin providing a feature like scene editing or animation graph editing.

You'd add the crate to the toml, add the plugin, and via the APIs it would add stuff to the command palette, and it would create a tab in the 2nd window that it uses to display it's stuff.

Something like this would mean the editor runs in game runtime. This makes it very easy for the editor to interact with the game world with little to no new APIs needed.

[alice-i-cecile]

Yep, definitely an interesting topic to explore (and I'd like multimonitor support too!). I think that's more in the realm of a potential toolkit, rather than the MVP abstraction however.

[codercommand]

Thought it may cover the abstraction since the dev tool might need to specify if it uses a tab or grid slot on the game window.

Edited: Nvm, it's answered in the RFC and I missed the point the first time. Why not add information about layout to our traits?

[naomijub]

1. I have the same concern as @rewin123, being someone that has dedicated my time to develop space editor UI, I feel that this solution is great and quite versatile, but doesn't address my main area of work "How this dev tools will be represented in the ui?". Besides what Rewin already elaborated on shared ui locality, I find that accessing dev_tool information and manipulating it in runtime is quite important, but it seems the focus was parsing and enabling, while I would like something more potent on the Reflection side, like hooks to events, systems, etc.

2. I also feel that storing keys as String is quite bothersome, specially for debugging purposes (HashMap<String, ...> ), this because we cannot easily preview or understand what are the contents, on the other hand, if we stored as TypeId, or something similar, the amount of options we can consider is vastly reduces. For example, String could be anything, but if we have a well organized project, TypeId, would be limited to modules named dev_tools or settings.

3. I feels that modals are equivalent to Resources and Command to Events, which is basically how we have implemented dev tools in space editor. I kind of expect a bit more from this kind of thing than what we already have with a tweak., but I dont have any constructive suggestions here.

[ricky26]

@alice-i-cecile / @rewin123 future considerations RE: layout is input redirection / modality. Might be kinda bike-sheddy.

The free-cam example seems obviously modal, and I would expect it to capture input (to avoid binding collisions with gameplay systems). (I do wonder whether we want to let the developers determine at which point input is captured, but I think this is a wider issue in bevy as it stands.)

For things like overlays (which is more where layout comes into play), I wouldn't expect them to be 'modal', since that usually refers to whole-window + input capture. We could just have an orthogonal system for allocating dev overlay space, and have these kind of things be DevCommands?

If you toggle multiple dev tools, which should get the input? (I'm expecting a lot of these tools will have overlapping bindings with gameplay, and each other.)

[alice-i-cecile]

Yeah, the focus management thing is generally a big open question in Bevy as a whole: see https://github.com/bevyengine/bevy/issues/3570 for more context there. I think it can and should be tackled independently though, and done ad-hoc to start.

[matiqo15]

I've been experimenting with abstraction for the dev tools for some time now and:

1. I still don't think type registry is a good idea:

From my understanding iter_with_data would require DevTool to be object-safe otherwise, we would never be able to get it, even as a &dyn Reflect. This is because we would need to use a ReflectDevTool to check whether it is a dev tool.

Even if we made DevTool object-safe, we would still run into problems getting &dyn DevTool from ReflectDevTool without type annotation.

    // New method on `TypeRegistry`
    pub fn iter_with_data<T: TypeData>(&self) -> impl Iterator<Item = (&TypeRegistration, &T)> {
        self.registrations.values().filter_map(|item| {
            let type_data = self.get_type_data::<T>(item.type_id());
            type_data.map(|data| (item, data))
        })
    }
    
    // Possible implementation of printing name of every tool
    for dev_tool in type_registry.iter_with_data::<ReflectDevTool>() {
        // This is the problem - we need to specify a tool here if we want to get a `&dyn DevTool`
        let reflected = Box::new(DevFlyCamera::default());
        if let Some(tool) = dev_tool.1.get(&*reflected) {
            println!("{:?}", tool.name());
        }
    }

2. Why shouldn't it be object-safe

We would need to drop FromReflect, and as a result of this, we would run into problems with executing dev commands.

3. DevToolsRegistry

You don't want dev tools to become a new unbounded framework - but actually, our abstraction can restrict what fits in it. With a type registry, you can always register a type, but whether you would be able to register a tool in our registry is going to be decided by the API of the registry. Would it be opinionated? - yes, but I think it should.

[cart]

I'm not seeing any problems here. Your first two hangups have solutions:

trait DevTool: Resource {
    fn name(&self) -> &'static str;
}

#[derive(Clone)]
struct ReflectDevTool {
    get_reflect: fn(&World) -> Option<&dyn Reflect>,
    get: fn(&World) -> Option<&dyn DevTool>,
    from_reflect: fn(&dyn Reflect) -> Option<Box<dyn DevTool>>,
}

impl ReflectDevTool {
    fn get_reflect<'a>(&self, world: &'a World) -> Option<&'a dyn Reflect> {
        (self.get_reflect)(world)
    }

    fn get<'a>(&self, world: &'a World) -> Option<&'a dyn DevTool> {
        (self.get)(world)
    }

    fn from_reflect(&self, reflect: &dyn Reflect) -> Option<Box<dyn DevTool>> {
        (self.from_reflect)(reflect)
    }
}

impl<D: DevTool + Reflect + FromReflect> FromType<D> for ReflectDevTool {
    fn from_type() -> Self {
        Self {
            get_reflect: |world| world.get_resource::<D>().map(|d| d as &dyn Reflect),
            get: |world| world.get_resource::<D>().map(|d| d as &dyn DevTool),
            from_reflect: |reflect| {
                D::from_reflect(reflect).map(|d| {
                    let d: Box<dyn DevTool> = Box::new(d);
                    d
                })
            },
        }
    }
}

#[derive(Resource)]
pub struct DevFlyCamera {}

fn iter_dev_tools(world: &mut World, type_registry: &TypeRegistry) {
    // Possible implementation of printing name of every tool
    for (registration, dev_tool) in type_registry.iter_with_data::<ReflectDevTool>() {
        if let Some(tool) = dev_tool.get(world) {
            println!("{:?}", tool.name());
        }
        let reflect = DynamicStruct::default();
        let d = dev_tool.from_reflect(&reflect);
    }
}

[vladinator1000]

As I'm reading this spec, it sounds exactly what an editor would be used for? Like a temporary solution that alleviates the lack of editor conventions, e.g. command panel.

Regardless of this classification, dev tools are: not intended to be shipped to end users highly application specific enabled or operated one at a time, on an as-needed basis, when problems arise intended to avoid interfering with the games ordinary operation, especially when disabled expansive and ad hoc

Also hi everyone, I normally just lurk around while gathering courage to contribute 😅