Add Metal support to Gazebo for macOS #1225
Conversation
chapulina
added
macOS
srmainwaring
force-pushed
the
feature/ign-gazebo7-metal-demo
branch
from
e5533e2 to
5e39eee
Compare
srmainwaring
force-pushed
the
feature/ign-gazebo7-metal-demo
branch
from
936a002 to
b6e4aff
Compare
chapulina
added
GUI
chapulina
removed
the
needs upstream release
|
hi @srmainwaring , I have merged this PR on main branch, but some icons is missing while manipulate shape, should it shows colorful arrows? camera_sensor.sdf is work too, but 1-refresh and 2-pause icons does not render. sensors_demo.sdf does not work, falling in infinite loop. [Msg] Ignition Gazebo Server v7.0.0~pre1 [Msg] Loading SDF world file[/usr/local/Cellar/ignition-gazebo7/6.999.999~0~20220412/share/ignition/ignition-gazebo7/worlds/sensors_demo.sdf]. [Dbg] [Physics.cc:824] Loaded [ignition::physics::dartsim::Plugin] from library [/usr/local/Cellar/ignition-physics6/5.999.999~0~20220414/lib/ign-physics-6/engine-plugins/libignition-physics-dartsim-plugin.dylib] [Dbg] [SystemManager.cc:49] Loaded system [ignition::gazebo::systems::Physics] for entity [1] [Dbg] [Sensors.cc:455] Configuring Sensors system [Dbg] [Sensors.cc:374] SensorsPrivate::Run [Dbg] [SystemManager.cc:49] Loaded system [Loaded system [[Dbg] [Sensors.cc:354] ignition::gazebo::systems::SensorsSensorsPrivate::RenderThread started] for entity [ ]] Waiting for init [Msg] Create service on [/world/lidar_sensor/create] [Msg] Remove service on [/world/lidar_sensor/remove] [Msg] Pose service on [/world/lidar_sensor/set_pose] [Msg] Pose service on [/world/lidar_sensor/set_pose_vector] [Msg] Light configuration service on [/world/lidar_sensor/light_config] [Msg] Physics service on [/world/lidar_sensor/set_physics] [Msg] SphericalCoordinates service on [/world/lidar_sensor/set_spherical_coordinates] [Msg] Enable collision service on [/world/lidar_sensor/enable_collision] [Msg] Disable collision service on [/world/lidar_sensor/disable_collision] [Msg] Material service on [/world/lidar_sensor/visual_config] [Msg] Material service on [/world/lidar_sensor/wheel_slip] [Dbg] [SystemManager.cc:49] Loaded system [ignition::gazebo::systems::UserCommands] for entity [1] [Dbg] [SystemManager.cc:49] Loaded system [ignition::gazebo::systems::SceneBroadcaster] for entity [1] [Msg] Loaded level [3] [Msg] Serving world controls on [/world/lidar_sensor/control], [/world/lidar_sensor/control/state] and [/world/lidar_sensor/playback/control] [Msg] Serving GUI information on [/world/lidar_sensor/gui/info] [Msg] World [lidar_sensor] initialized with [1ms] physics profile. [Msg] Serving world SDF generation service on [/world/lidar_sensor/generate_world_sdf] [Msg] Serving world names on [/gazebo/worlds] [Msg] Resource path add service on [/gazebo/resource_paths/add]. [Msg] Resource path get service on [/gazebo/resource_paths/get]. [Msg] Resource paths published on [/gazebo/resource_paths]. [Msg] Server control service on [/server_control]. [Msg] Found no publishers on /stats, adding root stats topic [Msg] Found no publishers on /clock, adding root clock topic [Dbg] [SimulationRunner.cc:494] Creating PostUpdate worker threads: 3 [Dbg] [SimulationRunner.cc:505] Creating postupdate worker thread (0) [Dbg] [SimulationRunner.cc:505] Creating postupdate worker thread (1) [Dbg] [Sensors.cc:541] Initialization needed [Msg] Loading plugin [ignition-rendering-ogre2] [Dbg] [RenderUtil.cc:2536] Create scene [scene] [Dbg] [Sensors.cc:577] Initialization needed [Dbg] [Sensors.cc:234] Initializing render context [Dbg] [Sensors.cc:253] Rendering Thread initialized [Msg] Serving scene information on [/world/lidar_sensor/scene/info] [Msg] Serving graph information on [/world/lidar_sensor/scene/graph] [Msg] Serving full state on [/world/lidar_sensor/state] [Msg] Serving full state (async) on [/world/lidar_sensor/state_async] [Msg] Publishing scene information on [/world/lidar_sensor/scene/info] [Msg] Publishing entity deletions on [/world/lidar_sensor/scene/deletion] [Msg] Publishing state changes on [/world/lidar_sensor/state] [Msg] Publishing pose messages on [/world/lidar_sensor/pose/info] [Msg] Publishing dynamic pose messages on [/world/lidar_sensor/dynamic_pose/info] [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [CameraSensor.cc:294] Camera images for [cameras_alone::link::cameras_alone] advertised on [camera_alone] [Dbg] [CameraSensor.cc:589] Camera info for [cameras_alone::link::cameras_alone] advertised on [/camera_info] [Dbg] [CameraSensor.cc:294] Camera images for [camera_with_lidar::link::camera] advertised on [camera] [Dbg] [CameraSensor.cc:589] Camera info for [camera_with_lidar::link::camera] advertised on [/camera_info] [Dbg] [DepthCameraSensor.cc:287] Depth images for [cameras_alone::link::depth_camera1] advertised on [depth_camera] [Dbg] [CameraSensor.cc:589] Camera info for [cameras_alone::link::depth_camera1] advertised on [/camera_info] [Dbg] [DepthCameraSensor.cc:304] Points for [cameras_alone::link::depth_camera1] advertised on [depth_camera/points] [Dbg] [RgbdCameraSensor.cc:222] RGB images for [rgbd_camera::link::rgbd_camera] advertised on [rgbd_camera/image] [Dbg] [RgbdCameraSensor.cc:236] Depth images for [rgbd_camera::link::rgbd_camera] advertised on [rgbd_camera/depth_image] [Dbg] [RgbdCameraSensor.cc:250] Points for [rgbd_camera::link::rgbd_camera] advertised on [rgbd_camera/points] [Dbg] [CameraSensor.cc:589] Camera info for [rgbd_camera::link::rgbd_camera] advertised on [rgbd_camera/camera_info] [Dbg] [Lidar.cc:122] Laser scans for [camera_with_lidar::link::gpu_lidar] advertised on [lidar] [Dbg] [GpuLidarSensor.cc:149] Lidar points for [camera_with_lidar::link::gpu_lidar] advertised on [lidar/points] [Dbg] [ThermalCameraSensor.cc:223] Thermal images for [thermal_camera::link::thermal_camera] advertised on [thermal_camera] [Dbg] [CameraSensor.cc:589] Camera info for [thermal_camera::link::thermal_camera] advertised on [/camera_info] [Msg] Setting ambient temperature to 288.15 Kelvin and gradient to -0.0065 K/m. The resulting temperature range is: 0.750384 Kelvin. [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed ... infinite loop here !!! so I pressed Ctrl+C ... [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [Sensors.cc:541] Initialization needed ^C[Dbg] [SignalHandler.cc:141] Received signal[2]. [Dbg] [ServerPrivate.cc:110] Server received signal[2] [Dbg] [Sensors.cc:382] SensorsPrivate::Stop [Dbg] [Sensors.cc:368] SensorsPrivate::RenderThread stopped [Dbg] [Sensors.cc:541] Initialization needed [Dbg] [ign.cc:356] Shutting down ign-gazebo-server [Dbg] [SimulationRunner.cc:521] [Dbg] [SimulationRunner.cc:521] [Dbg] [SimulationRunner.cc:521] Exiting postupdate worker thread (1Exiting postupdate worker thread ()0)0 ) ) [Dbg] [Sensors.cc:382] SensorsPrivate::Stop |
The issue with the Transform Control visuals is due to the QML gamma adjustment patch . The main branch in gz-gui uses a Qt extension QtGraphicalEffects, unfortunately it doesn't include Metal shaders so the patch I provided is a cheap and cheerful partial implementation. A full version would require reimplementing more of the Qt library - PR's welcome ;-). I don't have the issue you're seeing with the icons for the play / pause or refresh on the Image Display. The I'm using main for all the dependent libraries except gz-gui where I have my gamma adjust branch checked out, and gz-gazebo where I have this branch checked out (not merged into main - although it's current so that shouldn't make a difference?) |
I see, at least the function is work. I thought those icons miss is related to font or encoding. At first, I'v just want to install gazebo and cooperate with ros. but the brew version ignition-gazebo6 did not work, then I installed ignition-gazebo7, it still did not work, until I found your excellent PRs to support gazebo on macOS. my macOS version is 10.13.6, it have nVidia gtx1060 GPU card installed, so I have to stick on high sierra, the good thing is I can run some CUDA related lib, tensorflow, opencv. |
@srmainwaring any chance you could test this against OGRE2.3? I have a bottle in progress here: osrf/homebrew-simulation#1956 and a PR in flight here: gazebosim/gz-rendering#553 |
@mjcarroll sure, I'll take a look. I did have Metal running against an earlier version of the ogre2.3 PR so I'll bring those builds up to date. Update @mjcarroll this looks ok against OGRE2.3 with the minor change to one of the |
|
I came back to this PR and tested this after merging with Here are some general notes on testing gz-sim on macOS. All these issues are minor and not caused by this PR.
|
Thanks @iche033. I'm running a rebased version of this PR against Other notes: Some residual glitches on macOS in addition to those mentioned above - again not major and not connected to this PR.
Could that be connected to the z-order issue mentioned above? |
|
Thanks for dropping the notes on macOS issues. I'll create a ticket to track them
that works for me.
I see, I haven't tried setting other values. I should give that a try.
I tested mouse clicking and camera movement in gz-rendering examples and that worked fine. It could be mouse events issue in our gui. |
Update ign command script - Remove server only setting for macOS Enable the sensors server plugin to run on macOS - Initialise the render engine for sensors on the main thread - Platform specific OpenGL context code must be moved elsewhere. - Ensure the active OpenGL context is associated with the render thread post initialisation Enable the Gazebo GUI to run on macOS - Move the RenderThread object to the render thread at the same time as a the OpenGL context Reinstate render synchronisation - Reinstate Ignition / QML serial rendering IgnRenderer::Render. - Allow IgnRenderer::Render to accept a nullptr so it can be called without render sync from the main thread before the render thread is started. Update QML - Add missing properties for tooltopDelay Configure the Qt scene graph to use the Metal backend - Force the gazebo application to use the Metal backend for the Qt scene graph - Move Metal scene graph backend to ign::gui::Application Modify sensors to use Metal - Set the render system parameter to use Metal Default to Metal on macOS, otherwise use OpenGL - Add preprocessor checks to use the Metal graphics API for __APPLE__ Signed-off-by: Rhys Mainwaring <[email protected]>
…ent in one terminal Signed-off-by: Rhys Mainwaring <[email protected]>
- Add #ifdef to ensure sensors are initialised on the main thread for macOS Signed-off-by: Rhys Mainwaring <[email protected]>
Signed-off-by: Rhys Mainwaring <[email protected]>
srmainwaring
force-pushed
the
feature/ign-gazebo7-metal-demo
branch
from
8b3f4ff to
f852c1f
Compare
|
@iche033 - updated with requested changes and rebased. |
Signed-off-by: Rhys Mainwaring <[email protected]>
🎉 New feature
Fixes #44
Fixes #960
Summary
This PR provides support for running Gazebo on macOS using the Metal graphics API.
For platforms other than macOS the default behaviour is unchanged and will use the OpenGL graphics API. On macOS the graphics API is set to Metal and only the ogre2 render engine is supported.
Details
igncommand scriptsThe ruby script has been updated to allow
ign gazebo -gto be run on macOS. Because of macOS's rules about usingforkthe server and gui must be run in separate terminals.C++ code
There are two changes to support Metal
src/rendering/RenderUtil.cc: if__APPLE__is defined the render engine parameters are set tometal.src/systems/sensors/Sensors.cc: the render engine initialisation is moved toUpdateso that it occurs on the main thread.QML
Finally there are a number of small changes to QML files to address warnings and errors that show up up on macOS where we are using Qt 5.15.2_1. The are mostly to do with tool tip behaviour.
Test it
A document describing how to build Ignition for Metal support is here: https://github.com/srmainwaring/ign-rendering/wiki. The main changes to note are the dependencies are now:
and the MinimalScene plugin element to override the default graphics interface in
gui.configis now<graphics_api>metal</graphics_api>.examples/worlds/shapes.sdfUpdate the MinimalScene plugin element in
~/.ignition/gazebo/7/gui.configto include<graphics_api>metal</graphics_api>.Run the server:
Run the client:
examples/worlds/camera_sensor.sdfTwo changes are required in
camera_sensor.sdf:Then rebuild with
colcon(to install the change) and run as above substitutingcamera_sensor.sdfforshapes.sdfTests
These test are failing on macOS:
However
INTEGRATION_buoyancy_engineandINTEGRATION_touch_pluginpass when run individually.Code check
There are quite a few code check failures on macOS, however these don't seem to be connected to this PR.
Checklist
codecheckpassed (See contributing)Note to maintainers: Remember to use Squash-Merge