Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embed game process in editor #99010

Merged
merged 1 commit into from
Dec 20, 2024

Conversation

Hilderin
Copy link
Contributor

@Hilderin Hilderin commented Nov 10, 2024

Implements game embedding for Windows and Linux (X11 only) in the new Game Workspace: GitHub PR #97257

As suggested, this implementation places the running game window as a child of the editor and embeds it inside the Game Workspace. The running game remains a separate process, with all keys and inputs handled in a separate event loop to maintain performance even when embedded.

image

image

U7w7iZ8zil.mp4

New Options in the Game Workspace Toolbar

  • Embed game on Next Play: On/Off: Enables or disables embedding the next time the game is started. (Default: Enabled)

    • If multiple instances are configured, only the first instance will be embedded.
  • Make Game Workspace floating on Next Play: On/Off: When enabled, the Game Workspace opens in a floating window when the game starts. (Default: Enabled)

    • This option is not available when "Embed game on Play" is Off.
    • This option is not available in single-window mode.
  • Keep the aspect ratio of the embedded game: On/Off: Maintains the aspect ratio of the game window in the Game Workspace while embedding is enabled. (Default: Enabled)


Important Information

  • When embedded, some display settings cannot be changed at runtime to prevent the game window from exiting the Game Workspace. These settings will generate an error if modified during runtime in embedded mode:
    • Window Mode
    • Size
    • Minimum size
    • Maximum size
    • Position
    • Resizable
    • Always on Top
    • Popup
    • Current screen

Additional Features

  • Added the Engine.is_embedded_in_editor() method in GDScript/C#, which helps prevent errors when attempting to change unsupported window settings while embedded or for adjusting behavior based on whether embedding is active.
    • The same information can also be retrieved using OS.has_feature("embedded_in_editor").

Making Your Game "Embedded-Compatible"

  • Handling Mouse Capture: Currently, no default keyboard shortcut exists to exit mouse capture mode when embedded, which can be inconvenient for games like the TPS Demo that hide the mouse cursor during gameplay. As a workaround, you can modify the game to toggle the mouse cursor on pressing the Escape key instead of returning to the menu. Here’s an example for level.gd:

    func _input(event):
        if event.is_action_pressed("quit"):
            if Engine.is_embedded_in_editor():
                if Input.get_mouse_mode() == Input.MOUSE_MODE_VISIBLE:
                    Input.set_mouse_mode(Input.MOUSE_MODE_CAPTURED)
                else:
                    Input.set_mouse_mode(Input.MOUSE_MODE_VISIBLE)
            else:
                emit_signal("quit")
  • Preventing Fullscreen Errors: In the TPS Demo, errors occur when starting the game in fullscreen mode while embedded, as the DisplayServer does not allow window mode changes in embedded mode. You can prevent these errors using a simple check:

    if !Engine.is_embedded_in_editor():
        get_window().mode = Settings.config_file.get_value("video", "display_mode")

Known Issues

  • Windows: If the user focuses on the embedded game and quickly clicks a button in the editor, the click may not register. This seems to be due to Windows taking too long to reactivate the editor window, causing the mouse-up event to be missed. Disabling Unfocused Low Power Mode while the game is embedded mitigates this issue but is not a perfect solution.

  • Linux: X11 does not support moving windows outside screen bounds. If the editor window is moved outside the screen boundaries while embedding a game, the window is resized to prevent Linux from resetting its position to the screen edge. This limitation appears to be a known issue without a programmatic workaround.

  • Scene Previews Disabled: When the Game Workspace tab is active, scene previews are disabled because they appear under the embedded game. Since previews are standard controls rather than popups, this is a temporary workaround to prevent issues, though it disables scene previews. Addressing this in a future PR is recommended.

  • Game Process Recording: Tools like OBS Studio or Game Bar cannot record the embedded game by capturing the Godot Editor process. To record the game, you must capture the entire screen or a specific section of it. This limitation is expected given the separate processes for the editor and game.

  • Single Window Mode and Popups/Tooltips: When the game is embedded and the editor runs in single-window mode, all popups and tooltips are displayed beneath the embedded game process.

  • Embedded window can be moved with OS shortcuts: When the game is embedded some Windows or Linux shortcuts came be used to move the game window. I was not able to find a way to disable those shortcuts. On Windows we are talking about WindowKey+Shit+Right/Left and on Linux WindowKey+LeftMouseButton.


Testing

  • Windows: Tested on Windows 11 (1, 2, and 3 monitors) and Windows 10 (1 monitor).
  • Linux: Tested on Ubuntu 24.04 (1 and 2 monitors) and Fedora 40 KDE Plasma 6.1.5 (1 and 2 monitors).

@SheepYhangCN
Copy link
Contributor

Added the Engine.is_embedded() method in GDScript/C#

Add embedded into has_feature might be a better solution to ensure consistency since it is similar to the feature tags like debugor editor?

@AThousandShips AThousandShips changed the title Embedding game process in editor Embed game process in editor Nov 10, 2024
scene/scene_string_names.h Outdated Show resolved Hide resolved
servers/display_server.h Outdated Show resolved Hide resolved
@KoBeWi
Copy link
Member

KoBeWi commented Nov 10, 2024

If multiple instances are configured, only the first instance will be embedded.

Is this a limitation? If not, you could embed multiple instances inside a TabContainer.

EDIT:
Also the WINDOW_FLAG_HIDDEN allows for making apps that minimize to tray I think? (we already support tray icons)

@KoBeWi

This comment was marked as resolved.

@Hilderin Hilderin force-pushed the embedding-game-process branch from 9b9207e to abeae2c Compare November 10, 2024 11:48
@Hilderin
Copy link
Contributor Author

Add embedded into has_feature might be a better solution to ensure consistency since it is similar to the feature tags like debugor editor?

I added "embedded" to OS.has_feature to returns true when the game is running embedded. I still kept Engine.is_embedded to be consistent with Engine.is_editor_hint. Sounds good?

@Hilderin Hilderin force-pushed the embedding-game-process branch from abeae2c to c34b874 Compare November 10, 2024 12:05
@Meorge
Copy link
Contributor

Meorge commented Dec 5, 2024

Someone linked me this recent Bluesky post: https://bsky.app/profile/akien.bsky.social/post/3lckoohfoms2q

Only Windows and Linux for now [have embedded windows], but another contributor will tackle macOS next.

Is there a specific contributor actively working on this for now, or is this just meant to say "someone will work on it soon hopefully"? If the former, is there a place where I can watch progress?

@akien-mga
Copy link
Member

Is there a specific contributor actively working on this for now, or is this just meant to say "someone will work on it soon hopefully"? If the former, is there a place where I can watch progress?

See the above comment by @bruvzg, who's maintaining the macOS platform:

Is it due to technical limitations that it cannot be implemented on macOS, or are there other issues?

Directly embedding windows like it is done for X11 and Windows is not possible on macOS.

Eventually it will be supported, but will require a more complex approach (shared IOSurface to display game frame buffer and IPC to route input events from the editor to the game process).

@Meorge
Copy link
Contributor

Meorge commented Dec 5, 2024

See the above comment by @bruvzg, who's maintaining the macOS platform

Ah alrighty, thanks! I'd seen that comment previously, and interpreted it as more of a "it'll happen sometime by someone, but these are the hurdles they'd have to overcome from my knowledge/experience" than an explicit "I'll be working on it soon".

@aryan-11825114
Copy link
Contributor

Hey there, thanks for the amazing work! I just wanted to ask if this could make it to the next dev release?

@akien-mga
Copy link
Member

@Hilderin This will need another rebase.

@Hilderin Hilderin force-pushed the embedding-game-process branch from 1706e21 to 0fe5317 Compare December 13, 2024 03:53
@Hilderin
Copy link
Contributor Author

There you go... Rebase done!!

Copy link
Member

@YeldhamDev YeldhamDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get this when switching to the "Game" tab:

E 0:00:00:0127   window_set_position: Embedded window can't be moved.
  <C++ Error>    Condition "wd.embed_parent" is true.
  <C++ Source>   platform/linuxbsd/x11/display_server_x11.cpp:2265 @ window_set_position()

Fedora GNU/Linux 41 (XWayland).

editor/plugins/game_view_plugin.cpp Show resolved Hide resolved
editor/plugins/game_view_plugin.cpp Outdated Show resolved Hide resolved
editor/plugins/game_view_plugin.cpp Outdated Show resolved Hide resolved
@Hilderin
Copy link
Contributor Author

@YeldhamDev

I get this when switching to the "Game" tab:

E 0:00:00:0127   window_set_position: Embedded window can't be moved.
  <C++ Error>    Condition "wd.embed_parent" is true.
  <C++ Source>   platform/linuxbsd/x11/display_server_x11.cpp:2265 @ window_set_position()

I'm unable to reproduce this error. I tested on Fedora and Ubuntu. Can you share a MRP and explain with a bit more details how to reproduce the error? Thanks.

@Hilderin Hilderin force-pushed the embedding-game-process branch from 0fe5317 to 49b8034 Compare December 16, 2024 15:15
@YeldhamDev
Copy link
Member

I can reproduce it even with a fresh empty project, not doing anything special.

@geowarin
Copy link
Contributor

It works fine for me on archlinux/KDE (xwayland) 🤔

@Hilderin
Copy link
Contributor Author

@YeldhamDev

I can reproduce it even with a fresh empty project, not doing anything special.

Can you share your editor settings file, maybe you have a setting that was not tested/expected.

@YeldhamDev
Copy link
Member

@Hilderin editor_settings-4.zip

@Hilderin Hilderin force-pushed the embedding-game-process branch from 49b8034 to d968302 Compare December 17, 2024 21:38
@Hilderin
Copy link
Contributor Author

@YeldhamDev
Thanks for the editor settings file. Unfortunately, I was not enable to reproduce with your settings file neither. I looked every calls to window_set_position and I added a check on the one I think could be the issue. Can you try with this new build. If it's still not working, I would appreciate if you could debug the issue on your side to pinpoint the source of the issue since I cannot reproduce myself. It's a little tricky to debug the running embedded game. I suggest you start the game from inside the editor while the editor is not in debug. Copy the command line arguments from a processes monitor and use them to start a debug session while the editor is still running. You should see in the arguments a --wid followed by the x11 window id of the editor. Thanks.

@YeldhamDev
Copy link
Member

Sorry for the late response, was tangled in something else. Could you rebase?

@nubels
Copy link

nubels commented Dec 18, 2024

Fantastic work! I'd really love to see this in Godot 4.4. I hope there will be another dev snapshot.

@Hilderin Hilderin force-pushed the embedding-game-process branch from d968302 to 9d2a4c0 Compare December 18, 2024 22:52
Copy link
Member

@YeldhamDev YeldhamDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No errors on my end now, good job! I think there are still some stuff that could use some polish, but nothing deal breaking. LGTM.

@akien-mga akien-mga merged commit a11364d into godotengine:master Dec 20, 2024
20 checks passed
@akien-mga
Copy link
Member

Thanks for the amazing work @Hilderin, this complements the new Game tab very well!
Let's get this well-tested over the Christmas break so we can iron things out for 4.4 beta 1 in early January.

@stuartcarnie
Copy link
Contributor

stuartcarnie commented Dec 21, 2024

@akien-mga / @bruvzg OpenEmu does what you describe for macOS, as it spawns a child XPC process, and uses a remote layer (the Metal / Vulkan frame buffer), to display in the parent process, which is the most efficient. WebKit does this, by running subprocesses for the browser engine and displaying them all in a single, parent process.

I might take a look at that, if no one else is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.