Add support for embedding the game window in the editor

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Several users who want to switch to Godot (or switched to Godot already) miss the functionality provided by engines such as Unity for "exploring" the running game.

While Godot can show you all the running game contents in the remote scene tree and its sort of possible to override the remote camera, it still leaves a lot to be desired.

Engines like Unity or Unreal let you run the game within the editor window. They achieve this by running inside the same editor process, which lets you explore the game while you run it.

This approach has the clear advantage of allow you to explore the running game more intuitively, but it also has several disadvantages:

• Non-destructive editing of the running game is impossible or in a very limited fashion.
• The game has to be saved, the cleared before playing.
• The game has to be reloaded after playing.
• The editor stability can be easily compromised if the running game crashes.

None of these are problems with Godot however. Godot runs the game in a separate process and lets you explore its contents via IPC, but as the game window is not embedded, exploring the content becomes more unintuitive and annoying.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The idea is to have an embedded game mode in Godot. This proposal describes specifically how this could be done in a way that is feasible and should allow most of the expected behavior.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The main goal is to have a new mode in the top bar called "Game", where the game can run embedded in the editor. It would look something like this:

(Note: Pretend that the icon in game is a joystick icon and I am a very skilled graphic designer).

The idea is that the game runs in this window, at either native resolution or letting you stretch the window to fit it (zooming options can be discussed further, but you get the idea).

What functions are supported and what are those icons?

• Pause: Ability to pause the running game
• Next Frame: Ability to move to the next frame while paused.

• ---

• 2D/3D selection mode: This lets you select which kind of nodes and view will be interacted with while exploring the scene.

• ---

• Game input: When this icon is toggled and the game is focused (click on it to focus), all input is captured and goes to the game until user clicks elsewhere. (Of course if the game captures the mouse, we will need some way to un-capture it with some special keybinding).
• Selection Mode: This mode no longer sends input to the game and it lets you select 2D/3D nodes (depending on the selection mode described before) by clicking on them. They will appear on the remote scene tree and inspector so you can see their content.
• List Selection Mode: Same as the mode above, but it presents a list of nodes you can choose to select instead of selecting the topmost one.

• ---

• Camera override: This lets you override the game camera and pan/zoom (if 2D is selected) or WASD/similar (if 3D is selected).

Implementation notes:

The main idea of this feature is that Godot will remain running the game as a sub process, with the main difference being that the game window is embedded into the editor. This can be done with relevant Vulkan extensions (and worse case if a platform does not support this, it just does not let you embed the game window, but most should support it).

FAQ:

Q: Are there not other 10 proposals about this? A: Yes, but this explains how it would be done in a way that is feasible technically.

Q: Will the 2D and 3D editors show the game objects moving around? A: No, this is not possible and probably never will. Exploring will have to happen from the game window itself. We can add tools to make this as comfortable as possible.

Q: Will I still be able to edit the scene while the game runs in a non-destructive way? A: Yes, unlike other engines, you will not lose any change made to the scenes you are editing and the changes will also replicate to the running game.

Q: Will the editor stability be compromised by the game? A: No, it should not be.

Q: Is this feature optional? A: This feature is not only optional but probably only present for platforms and rendering drivers where it is possible to do this kind of embedding. I suppose users will be able to configure whether they want to use it by default or not in the editor settings.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This is core

Is there a reason why this should be core and not an add-on in the asset library?

N/A

[aaronfranke]

This would significantly increase the usability of the editor camera override. Currently you need to have both the editor and the game visible on screen, then enable the camera override, and you need to have the editor focused to move the camera in the game. This proposal significantly improves the situation so that you only need the editor visible.

[yosoyfreeman]

Could be an option to not save the changes made while running the game in this mode? I personally miss it to quickly test things, spawn objects or change values being sure it will revert when i stop it.

[aaronfranke]

@yosoyfreeman You would not be spawning objects into the running game directly. You can only spawn objects in the saved scenes, and in many cases Godot can sync that to the running game. This proposal does not change this behavior, it only improves the usability by allowing the game window to embed into the editor.

[bitbrain]

I really like this proposal! How would this behave if I were to set a breakpoint somewhere? I suspect since it still runs in a sub-process, the game would be set to paused if it reached a breakpoint?

[Bastienre4]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

[BoneDaddy64]

Coming from Unity, all I really want is to have a scene view (probably should be embedded) that has a separate camera from the game view but renders the same scene state, as in same objects (transforms, models, animation state etc. Unity allows for different shading which is good) that I can fly around with, and to be able to pick (as in click in that view on the rendered object's model) and have the Inspector show that node (and be able to change it, like when in Remote in the Scene list currently). Whether the game view is embedded or not doesn't matter that much to me.

Basically I just want to be able to quickly fly around, select objects, have those objects appear in the Inspector, be able to real-time modify the objects (which Godot has the functionality for now). That's what's really lacking right now for me in Godot, makes it really cumbersome to "touch" my game as it's running

[aaronfranke]

@BoneDaddy64 Depending on the details of how the implementation of copying the texture to the editor works, it might be possible to have the editor camera override add a new viewport + camera and read the texture from there, instead of overriding the existing camera's transform, so the game view could keep the same camera. That would be a separate proposal made after this one, but it would help if this feature was designed with that use case in mind.

[allenwp]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

This comment about gizmos brings up the question of whether gizmos could be drawn over top of this new Game view mode to visually show what is selected. Further to this, there are questions about whether these gizmos could modify the remote node properties, such as using the transform gizmos to translate, rotate, or scale remote nodes.

Personally, I suspect that this is something that should be kept in mind when implementing this new feature so that full gizmo support could be added at a later time. I think it would be great to get something like this implemented without gizmo overlays sooner, rather than delaying the feature to have full-featured gizmo overlays.

I also think that just being able to see gizmos in the scene would be immensely helpful to visually distinguish what node is selected and shown in the inspector panel. Being able to modify the node's properties using gizmos is a less valuable feature because this can be done through the inspector panel.

[sosasees]

i need to test window transformations often — like moving and resizing — especially when they are not triggered in the OS but in the game.

so the current way of running the game in its own window saves me much time.

so if people want the game window embedded, this should be an option and not the only way.

[allenwp]

I previously started a discussion regarding "Spacial Editor - Focus Selection should focus on remote nodes". I think this new proposal should address this feature: When in the new Game view mode in "Selection Mode", pressing the F key to Focus Selection should focus the camera override on the currently selected remote node.

This is especially helpful when you have nodes that are far away from your game or override camera, such as an off-screen world object. With this Focus Selection feature implemented for the Game view mode, it would allow the user to select the node in the remote scene tree, Focus Selection, and have the override camera focus on this node.

I also noticed that the original post mockup screenshot does not show a remote scene tree view at all, and it seems that the proposal suggests a "List Selection Mode" instead. How does this List Selection Mode compare to the existing remote scene tree? I'm curious to know more about this, especially in the context of how it will interact with node selection and the Focus Selection action.

[m4gr3d]

This would be a welcome improvement for the Android editor, given the limitations on tablets and phones to have multiple windows open side by side.

[robbertzzz]

I'd personally really like if the UI would allow for switching between different performance views. I sometimes want to see wireframes or overdraw in my game, but can currently only do it by adding additional code to the game. Some UI for it like Unreal has would be amazing!

[NathanLovato]

Overall, it sounds great, and it's been requested time and time again, as you know. This is especially big for professionals coming from Unity and Unreal.

UX-wise, do you plan to unify the play scene functionality (the transport controls in the top-right) with this? Here are some thoughts to complement the proposal regarding making this view work with the existing transport control, UX-wise.

Unifying the transport controls with this new Game main screen:

• By default, playing the scene or game with F5 or F6 shows the running game in this new embedded Game main screen.
• Pressing F8 stops the game view and takes you back to the last main screen you were working with (2D or 3D mainly).
• An icon similar to the one to pop out the script editor is present on the Game main screen and lets you float the Game main screen view to another screen.
• When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)
• An icon on the Game view allows you to toggle it full-screen (to have this game view full-screen on another display, e.g. for two-people test sessions where one person plays the game with a gamepad and a developer or level design makes changes with hot reload).

These are just quick ideas to spark discussion, to see if that's possible or the intention at all.

The idea would be to unify the current play scene functionality with this new view, so that users have one consistent and powerful experience when running their scenes or games in the editor.

If we can do anything to help, please let me know. We now have an experienced UX/UI designer specialist in the team, we can help with mockups, icons, and focused user testing moving forward if the team needs extra hands.

[reduz]

@NathanLovato yeah it does make sense to do this if you are using the embedded mode.

[yosoyfreeman]

* By default, playing the scene or game with F5 or F6 shows the running game in this new embedded Game main screen.

* Pressing F8 stops the game view and takes you back to the last main screen you were working with (2D or 3D mainly).

* An icon similar to the one to pop out the script editor is present on the Game main screen and lets you float the Game main screen view to another screen.

* When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)

* An icon on the Game view allows you to toggle it full-screen (to have this game view full-screen on another display, e.g. for two-people test sessions where one person plays the game with a gamepad and a developer or level design makes changes with hot reload).

These are just quick ideas to spark discussion, to see if that's possible or the intention at all.

The idea would be to unify the current play scene functionality with this new view, so that users have one consistent and powerful experience when running their scenes or games in the editor.

As @sosasees said, the current workflow is solid and clean and should be maintained. I don't think is a good idea to default to this new mode and needing extra clicks to get to the floating window that was default. I think specific shortcuts for this mode would make more sense from a usability perspective, so if its not possible to disable it, at least do not interfere.

I think that the toolbar should remain where it is. Is the editor the one who play or stop scenes, so for me it makes more sense that is there, always in the same place.

I think this is good, specially for new users, but should not replace what is already there.

[Bastienre4]

The best answer to this "Should it be default or not" is to add an editor setting to define this. So that everyone is happy :)

[sosasees]

• When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)

there are multiple ways this could be done:

toolbar inside window title bar

• ⭕️ it's very clear that the toolbar belongs to the window

but it has several issues that make it a bad fit for Godot:

• ❌️ if users want to add their own buttons to their game's window title bar (if that ever becomes a feature), they can't see and use them in editor builds
• ❌️ borderless windows have no window title bar, so the toolbar disappears in borderless windows
• ❌️ if the window is too narrow, the toolbar has to be truncated with an overflow menu (and with extreme small window sizes like 64×64px, even the overflow menu becomes inaccessible because it's overlapped by the minimise/maximise/close buttons)
• ❌️ Windows (and probably iMac, i don't know enough about iMac to know for sure) have good support for additional window title buttons like this, but not Linux (at least not without hacks)

toolbar under window title bar

this seems similar to toolbar inside window title bar, but it's much better like this:

• ⭕️ much more room for window title
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ⭕️ toolbar is outside the window title bar, so it doesn't disappear in borderless window mode

of course, it still has some of the same advantages and disadvantages of toolbar inside window title bar:

• ⭕️ it's very clear that the toolbar belongs to the window
• ❌️ if the window is too narrow, the toolbar has to be truncated with an overflow menu (but this time the overflow menu stays accessible in extreme small window sizes like 64×64px because nothing overlaps)

and we need to be cautious of

• ❌️ Godot needs to subtract the toolbar height from the window height (just like it subtracts the window border) if we want to get the real game viewport height value

toolbar in own window

• ⭕️ toolbar does not truncate if the game window has an extreme small size like 64×64px
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ❌️ connection to the game window is more abstract
• ❌️ we need extra window management programming so that the toolbar window moves as expected:
  • if it has not been moved by the user since the game started, it should move together with the game window
  • if it automatically moves to a position where it goes (at least partially) off-screen, it should move down until it's fully visible
  • if it has been moved by the user, it should not move automatically anymore until the game is restarted
  • it should be always on top if the game window is focused

toolbar under window title bar, but as its own borderless window

¿why didn't i think of this much earlier? 🤦‍♂️ (i added it when editing the comment 3 days later)

this combines the best of toolbar under window title bar and toolbar in own window:

• ⭕️ it's very clear that the toolbar belongs to the window
• ⭕️ toolbar does not truncate if the game window has an extreme small size like 64×64px
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ⭕️ toolbar is outside the window title bar, so it doesn't disappear in borderless window mode
• ⭕️ we don't need much window management programming to keep the toolbar in place: just always move it to the same position relative to the window
• ❌️ Godot needs to subtract the toolbar height from the window height (just like it subtracts the window border) if we want to get the real game viewport height value

[NathanLovato]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

---

Note that I personally don't mind either, with what we do at GDQuest, we're not too concerned by discoverability problems, because we always keep up with the latest development in Godot. This point is about taking the time to consider the wider user-base, current and future, and thinking about what's the best experience for them.

[Bastienre4]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

Agreed. Also, I'm currently working on a single 21:9 display, and having the floating player window everytime I want to test the game is a pain if, as you said, I need to go back and forth in the editor.

[NathanLovato]

Bringing a good point/suggestion from a Twitter user: for mobile game developers, but also app developers, it's useful to be able to quickly test the view at different sizes matching popular mobile devices, to see if the game/UI scales properly on different displays.

So they pointed out that Unity, but we could add web browsers, offer a dropdown to quickly change the preview size to these devices' resolution.

This may be out of scope for this proposal, but seeing that the game view would be resizable, would it be possible to have a dropdown or something to select common presets? For various mobile phones, tablets, and typical desktop display ratios (16:9, 16:10, and the likes).

Please let me know if it should go into a separate proposal.

[yosoyfreeman]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

Note that I personally don't mind either, with what we do at GDQuest, we're not too concerned by discoverability problems, because we always keep up with the latest development in Godot. This point is about taking the time to consider the wider user-base, current and future, and thinking about what's the best experience for them.

I understand the importance of the first experience and fully agree with you. I may be failing to see the benefits tho. I think that what users miss the most is the fully editable and reversible playback and this that Reduzio pointed out:

Q: Will the 2D and 3D editors show the game objects moving around? A: No, this is not possible and probably never will. Exploring will have to happen from the game window itself. We can add tools to make this as comfortable as possible.

We would still missing the two more famous features of those engines. So in the end, there is still a learning curve towards a different approach.

Just to understand better, you talked about quickly moving the camera to the point and moving things and seeing it in game. Is this not that?

Grabación de pantalla desde 2023-07-07 11-28-51.webm

I do agree that some way to quickly test screen ratios would be great.

PS: Please, remember that i lack social skills and i apology if my tone is not the right one. I'm just trying to give the feedback you asked in an organized way.

[sosasees]

a dropdown to quickly change the preview size to these devices' resolution

some way to quickly test screen ratios would be great.

i just got a new add-on idea: window size panel

i will most likely not make this add-on because i only make add-ons that i need. otherwise i would make too many add-ons once and never update them to newer Godot versions.

[robbertzzz]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

Unreal has a drop-down menu next to the play button which allows you to pick how the play button behaves. I really like it because it's immediately clear to the user yet works as a project setting.

[lunarcloud]

As long as the current behavior (run it like for real) remains. It's actually something I really like.

[mmMarzban]

I want that!

[geekley]

To me, the most important feature here is this:

Selection Mode: This mode no longer sends input to the game and it lets you select 2D/3D nodes (depending on the selection mode described before) by clicking on them. They will appear on the remote scene tree and inspector so you can see their content.

Godot really needs a way to select nodes (instantiated at runtime) in the Game View by clicking, even if it's in the current implementation with separate window. In fact, I hope that when this is implemented, we can still use all those new features regardless of whether the game view is embedded or detached / separate window like now. If you're gonna implement this step-by-step, this would be the best thing to have first IMO, and if you wanted to back-port just 1 thing from here to 3.x, this would be the most helpful (and the one feature that has no workarounds). (I'm not asking to back-port btw, just saying that to say it's the most important feature)

List Selection Mode: Same as the mode above, but it presents a list of nodes you can choose to select instead of selecting the topmost one.

Hopefully it will also unify with previous feature without requiring the separate mode/button, with alt+right-click.

[geekley]

Camera override: This lets you override the game camera and pan/zoom (if 2D is selected) or WASD/similar (if 3D is selected).

I'm assuming this would allow to have the camera override in the Game View independently of the Scene View's camera, which is great. But you could also keep/reuse the current equivalent button in Scene View to decide if you still want to sync both view's "editor cameras" or move them independently.

[geekley]

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

Just to understand better, you talked about quickly moving the camera to the point and moving things and seeing it in game. Is this not that?

@yosoyfreeman Yes, but because you can't click to select remote nodes that you instantiate only at run-time, it's not easy to test bugs. In Unity you don't have this issue. That's why I think Selection Mode in Game View is the most important missing feature here, and IMO should be the one with highest priority among those listed.

[GeorgeS2019]

Not sure if someone written, this feature is good for unit testing scene without resort to godot.exe -scene testscene.tscn

[Calinou]

This can be done with relevant Vulkan extensions (and worse case if a platform does not support this, it just does not let you embed the game window, but most should support it).

If I'm understanding right, this means having to implement something akin to https://github.com/godotengine/godot-proposals/issues/5790.