Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support nested model #258

Merged
merged 26 commits into from
Oct 14, 2020
Merged

Support nested model #258

merged 26 commits into from
Oct 14, 2020

Conversation

iche033
Copy link
Contributor

@iche033 iche033 commented Jul 21, 2020

depends on gazebosim/gz-physics#86 and gazebosim/sdformat#316

This PR updates the physics system to support parsing and constructing nested models from Model SDF DOM (see gazebosim/sdformat#316).

Since the physics system relies heavily on canonical links for setting and also getting pose information of models in the physics engine, some changes were made to support the case where the canonical link is inside a nested model, e.g.:

<model name="top_level_model">
  <model name="nested_model">
      <link name="nested_link"> ... </link>
      ...
  </model>
</model> 

In the example above, the top level model has one nested model and no child links. In this case, the canonical link would be set to the link inside the nested model. A couple of helper functions (TopLevelModel and RelativePose) were added to help compute the transform from the top level model to the nested link in order to correctly set and get model poses.
Note that I considered using ign-physics's ModelFrameSemantics feature to compute this transform data but the feature is currently not part of the minimum feature list, so it's probably not a good idea to depend on an optional physics feature when computing pose updates for a core part of the physics system.

Here's an example nested model world:

ign gazebo -v 4 --physics-engine ignition-physics-tpe-plugin nested_model.sdf

@iche033 iche033 added enhancement New feature or request 🏰 citadel Ignition Citadel labels Jul 21, 2020
@iche033 iche033 requested review from azeey and chapulina as code owners July 21, 2020 04:29
@luca-della-vedova
Copy link
Member

CC @Kevinskwk to test with our nested model simulation environment.

@luca-della-vedova
Copy link
Member

I don't know if this is slightly outside the scope. From what I understand the support was only introduced for TPE (which is OK), however when running a world with nested models with DART the behavior changed from printing an error about nested models not being supported (before the PR), to a segmentation fault with the attached backtrace.
Is this intended? Should there be a feature in the physics engine to tell whether nested models are supported or not and avoid trying to spawn them if it is not supported?

Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00007fffbd095530 in dart::simulation::World::getNumSkeletons() const () from /usr/lib/x86_64-linux-gnu/libdart.so.6.10
(gdb) bt
#0  0x00007fffbd095530 in dart::simulation::World::getNumSkeletons() const () from /usr/lib/x86_64-linux-gnu/libdart.so.6.10
#1  0x00007fffbd6d5c46 in ignition::physics::dartsim::Base::AddModel (_worldID=26, _info=..., this=0x55555e7ec908)
    at /usr/include/c++/8/bits/shared_ptr_base.h:1018
#2  ignition::physics::dartsim::SDFFeatures::ConstructSdfModel(ignition::physics::Identity const&, sdf::v9::Model const&) ()
    at /home/luca/ws_citadel/src/ign-physics/dartsim/src/SDFFeatures.cc:359
#3  0x00007fffd8410a5c in ignition::physics::sdf::ConstructSdfModel::Model<ignition::physics::FeaturePolicy<double, 3ul>, ignition::gazebo::v4::systems::PhysicsPrivate::MinimumFeatureList>::ConstructModel (_model=..., this=0x7fffffffb208)
    at /home/luca/ws_citadel/install/include/ignition/plugin1/ignition/plugin/detail/SpecializedPlugin.hh:236
#4  ignition::gazebo::v4::systems::PhysicsPrivate::CreatePhysicsEntities(ignition::gazebo::v4::EntityComponentManager const&)::{lambda(unsigned long const&, ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > > const*, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer> const*, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > > const*, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > const*)#2}::operator()(unsigned long const&, ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > > const*, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer> const*, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > > const*, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > const*) const [clone .isra.2050] () at /home/luca/ws_citadel/src/ign-gazebo/src/systems/physics/Physics.cc:712
#5  0x00007fffd84594cd in std::function<bool (unsigned long const&, ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > > const*, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer> const*, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > > const*, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > const*)>::operator()(unsigned long const&, ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > > const*, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer> const*, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > > const*, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > const*) const (__args#4=<optimized out>, __args#3=<optimized out>, 
    __args#2=<optimized out>, __args#1=<optimized out>, __args#0=@0x7fffffffb380: 220, this=0x7fffffffb410)
    at /usr/include/c++/8/bits/std_function.h:682
#6  ignition::gazebo::v4::EntityComponentManager::EachNew<ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > >, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer>, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > >, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > >(ignition::gazebo::v4::EntityComponentManager::identity<std::function<bool (unsigned long const&, ignition::gazebo::v4::components::Component<std::add_lvalue_reference<void>, ignition::gazebo::v4::components::ModelTag, ignition::gazebo::v4::serializers::DefaultSerializer<std::add_lvalue_reference<void> > > const*, ignition::gazebo::v4::components::Component<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, ignition::gazebo::v4::components::NameTag, ignition::gazebo::v4::serializers::StringSerializer> const*, ignition::gazebo::v4::components::Component<ignition::math::v6::Pose3<double>, ignition::gazebo::v4::components::PoseTag, ignition::gazebo::v4::serializers::DefaultSerializer<ignition::math::v6::Pose3<double> > > const*, ignition::gazebo::v4::components::Component<unsigned long, ignition::gazebo::v4::components::ParentEntityTag, ignition::gazebo::v4::serializers::DefaultSerializer<unsigned long> > const*)> >::type) const (this=this@entry=0x555555c2c620, _f=...)
    at /home/luca/ws_citadel/src/ign-gazebo/include/ignition/gazebo/detail/EntityComponentManager.hh:348
#7  0x00007fffd8414f75 in ignition::gazebo::v4::systems::PhysicsPrivate::CreatePhysicsEntities(ignition::gazebo::v4::EntityComponentManager const&) () at /usr/include/c++/8/new:169
#8  0x00007fffd8415f38 in ignition::gazebo::v4::systems::Physics::Update(ignition::gazebo::v4::UpdateInfo const&, ignition::gazebo::v4::EntityComponentManager&) () at /home/luca/ws_citadel/src/ign-gazebo/src/systems/physics/Physics.cc:619
#9  0x00007ffff4a682d2 in ignition::gazebo::v4::SimulationRunner::UpdateSystems (this=this@entry=0x555555c2c520)
    at /home/luca/ws_citadel/src/ign-gazebo/src/SimulationRunner.cc:459
#10 0x00007ffff4a6dbd7 in ignition::gazebo::v4::SimulationRunner::Step (this=this@entry=0x555555c2c520, _info=...)
    at /home/luca/ws_citadel/src/ign-gazebo/src/SimulationRunner.cc:696
#11 0x00007ffff4a6e29b in ignition::gazebo::v4::SimulationRunner::Run(unsigned long) ()
    at /home/luca/ws_citadel/src/ign-gazebo/src/SimulationRunner.cc:669
#12 0x00007ffff4a62915 in ignition::gazebo::v4::ServerPrivate::Run(unsigned long, std::optional<std::condition_variable*>) ()
    at /usr/include/c++/8/bits/unique_ptr.h:345
#13 0x00007ffff4a5db11 in ignition::gazebo::v4::Server::Run(bool, unsigned long, bool) () at /usr/include/c++/8/optional:969
#14 0x00007ffff5184918 in runServer () at /home/luca/ws_citadel/src/ign-gazebo/src/ign.cc:267
#15 0x00007ffff5592dae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#16 0x00007ffff559271f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#17 0x00007ffff57998f8 in ?? () from /usr/lib/x86_64-linux-gnu/ruby/2.5.0/fiddle.so
#18 0x00007ffff7aa3cab in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#19 0x00007ffff579968d in ?? () from /usr/lib/x86_64-linux-gnu/ruby/2.5.0/fiddle.so
#20 0x00007ffff7aca3c9 in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#21 0x00007ffff7ad88f3 in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#22 0x00007ffff7aced85 in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#23 0x00007ffff7ad4b64 in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#24 0x00007ffff79af0c4 in ?? () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#25 0x00007ffff79b0f4d in ruby_exec_node () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#26 0x00007ffff79b342e in ruby_run_node () from /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5
#27 0x00005555555548cb in ?? ()
#28 0x00007ffff7554b97 in __libc_start_main (main=0x555555554880, argc=5, argv=0x7fffffffc7a8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffc798) at ../csu/libc-start.c:310
#29 0x00005555555548fa in _start ()

@iche033
Copy link
Contributor Author

iche033 commented Jul 22, 2020

I added a check in 7740d5c that to make sure the constructed model is not null and warned the user if it is. Note you'll also need to update the tpe_nested_model branch in ign-physics. This may be sufficient for now to tell whether or not the physics engine supports nested model. I think there are also plans to add nested model support to dartsim plugin in ign-physics and we could consider adding a nested model feature.

@luca-della-vedova
Copy link
Member

I added a check in 7740d5c that to make sure the constructed model is not null and warned the user if it is. Note you'll also need to update the tpe_nested_model branch in ign-physics. This may be sufficient for now to tell whether or not the physics engine supports nested model. I think there are also plans to add nested model support to dartsim plugin in ign-physics and we could consider adding a nested model feature.

Tried it out and indeed the crash is fixed, thanks! I'll try to get a full tpe sim up and running but that is quite a daunting task so don't keep it as a blocker to get this merged 😅

Copy link
Contributor

@claireyywang claireyywang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Just have some minor comments.

src/systems/physics/Physics.cc Outdated Show resolved Hide resolved
test/integration/physics_system.cc Show resolved Hide resolved
Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit torn about this. We're adding Model::ConstructModel inside a the ConstructSdfModel feature, but then using the same Implementation function with an extra check. This creates a situation where a feature is partially implemented by a physics engine. In this case, the DART plugin doesn't have this capability so it returns an invalid ID. Instead it would be nice to have a separate feature, maybe called ConstructSdfNestedModel so that other physics engines that don't support nesting can still be used when nesting is not required.

On the flip side, since the SDFormat composition proposal is still in progress, the newly created ConstructSdfNestedModel would have the same issue of being a partial implementation once the SDFormat proposal lands in libsdformat11. Nevertheless, my preference would be to create the new feature and document that it is not complete. This would be a notice to whoever wants to implement this feature that its requirements may change soon.

If we create the new feature, we can formally limit the ConstructSdfModel feature to work with non-nested models.

@iche033
Copy link
Contributor Author

iche033 commented Aug 1, 2020

Nevertheless, my preference would be to create the new feature and document that it is not complete.

ok I added ConstructSdfNestedModel feature in gazebosim/gz-physics#86 and updated the physics system to use this optional feature in 57c2d1d

If we create the new feature, we can formally limit the ConstructSdfModel feature to work with non-nested models.

I have not added this limitation yet. To confirm, this means we need to add ConstructSdfNestedModel feature to both WorldPtrType and ModelPtrType? Currently I use world->ConstructModel and model->ConstructNestedModel, both of them can consume an sdf::Model object with nested models. If we limit ConstructSdfModel feature to non-nested models, we would need to do an additional check on the input sdf::Model to see if it has child models before determining which Construct feature to use, correct?

@azeey
Copy link
Contributor

azeey commented Aug 5, 2020

To confirm, this means we need to add ConstructSdfNestedModel feature to both WorldPtrType and ModelPtrType? Currently I use world->ConstructModel and model->ConstructNestedModel, both of them can consume an sdf::Model object with nested models. If we limit ConstructSdfModel feature to non-nested models, we would need to do an additional check on the input sdf::Model to see if it has child models before determining which Construct feature to use, correct?

Yes, this is what I was thinking. @scpeters, does this sound reasonable?

src/systems/physics/Physics.cc Show resolved Hide resolved
const auto *staticComp =
_ecm.Component<components::Static>(_parent->Data());
_ecm.Component<components::Static>(topLevelModel);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be better to override the <static> element of child models during creation and checking the model's own <static> rather than going up the tree and fining the top level model. If, for example, a model is a grandchild whose parent is static=true, but whose grandparent (top level model) is static=false, this would not catch that.

More detail: gazebosim/sdf_tutorials#35

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I see, it seems like the proposal allows the child nested models to be static while the top level model can be non-static but not the other way around? So:

  • if top level model is static, all nested models in the tree will be static.
  • if top level model is non-static, nested models can be static or non-static

implemented in c070152

src/systems/physics/Physics.cc Show resolved Hide resolved
@iche033
Copy link
Contributor Author

iche033 commented Aug 8, 2020

Yes, this is what I was thinking. @scpeters, does this sound reasonable?

I added a new ContructedSdfNestedModel to WorldPtrType in gazebosim/gz-physics#86 and updated the physics system to make use of this new optional feature. c070152

Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm approving this PR, but I want to note that we need to address the issue of nested model poses at some point. This PR only updates the pose of the top level model and leaves the poses of nested models at their initial state. However, a number of plugins (eg. Breadcrumbs) rely on the model's pose in their computation, which means they will fail if the model they are in is nested inside another model. This is not an issue right now since TPE doesn't support joints and pose and velocity commands are disabled for nested models.

include/ignition/gazebo/SdfEntityCreator.hh Show resolved Hide resolved
auto ent = this->CreateEntities(_model, true, false);

// Load all model plugins afterwards, so we get scoped name for nested models.
for (const auto &[entity, element] : this->dataPtr->newModels)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a std::map for newModels will probably mean that the plugins of the parent model will be loaded before the plugins of the child model since the Entity ID of the parent will be a smaller number than the child's. Do you think that would be an issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mainly done so the all the ancestor models are loaded first before loading the nested model's plugin so that it's able to get the scoped name. As for the order of loading the plugins, I think it maybe fine but not 100% certain. I just checked the behavior in gazebo classic and it's also loading the parent model's plugin first before the children's plugins. So we could give this a try and come back if we run into issues in the future?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.

<< " engine doesn't support feature "
<< "[ConstructSdfNestedModelFeature]. "
<< "Nested model will be ignored."
<< std::endl;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a71a3d2 added the informed variable in the first entityCast above, but not here.

src/systems/physics/Physics.cc Show resolved Hide resolved
@@ -0,0 +1,110 @@
<?xml version="1.0" ?>
<sdf version="1.6">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 1.6 -> 1.7

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. 6b74143

Signed-off-by: Ian Chen <[email protected]>
@iche033
Copy link
Contributor Author

iche033 commented Sep 30, 2020

ok I think I addressed all comments and ticketed issue #392 about the nested model pose issue.

@iche033
Copy link
Contributor Author

iche033 commented Oct 2, 2020

@osrf-jenkins run tests please

@chapulina chapulina added needs upstream release Blocked by a release of an upstream library and removed needs upstream release Blocked by a release of an upstream library labels Oct 6, 2020
@scpeters
Copy link
Member

scpeters commented Oct 9, 2020

when finding the sdformat9 version in cmake, I think we should require version 9.3.0

@iche033
Copy link
Contributor Author

iche033 commented Oct 10, 2020

bumped sdformat version requirement to 9.3.0 in 44cd092

@chapulina chapulina removed the needs upstream release Blocked by a release of an upstream library label Oct 12, 2020
Signed-off-by: Ian Chen <[email protected]>
@codecov
Copy link

codecov bot commented Oct 14, 2020

Codecov Report

Merging #258 into ign-gazebo3 will increase coverage by 0.02%.
The diff coverage is 71.75%.

Impacted file tree graph

@@               Coverage Diff               @@
##           ign-gazebo3     #258      +/-   ##
===============================================
+ Coverage        77.19%   77.21%   +0.02%     
===============================================
  Files              203      203              
  Lines            10864    10963      +99     
===============================================
+ Hits              8386     8465      +79     
- Misses            2478     2498      +20     
Impacted Files Coverage Δ
include/ignition/gazebo/SdfEntityCreator.hh 100.00% <ø> (ø)
.../plugins/component_inspector/ComponentInspector.hh 28.57% <ø> (ø)
.../plugins/component_inspector/ComponentInspector.cc 9.48% <25.00%> (+0.38%) ⬆️
src/systems/physics/Physics.cc 70.74% <63.09%> (+0.43%) ⬆️
src/SdfEntityCreator.cc 90.52% <100.00%> (+0.48%) ⬆️
src/SdfGenerator.cc 96.00% <100.00%> (+0.09%) ⬆️
src/Util.cc 95.15% <100.00%> (+0.34%) ⬆️
src/systems/scene_broadcaster/SceneBroadcaster.cc 97.73% <0.00%> (+0.32%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ca9862...fc2d91d. Read the comment docs.

@iche033 iche033 merged commit b715943 into ign-gazebo3 Oct 14, 2020
@iche033 iche033 deleted the nested_model branch October 14, 2020 05:31
@chapulina
Copy link
Contributor

Ouch, this should have been squash-merged. Now all of this PR's commits are scattered around the ign-gazebo3 branch, which makes it hard to identify what changes came from each PR:

ignition-gazebo3_3.3.0...ign-gazebo3

@iche033
Copy link
Contributor Author

iche033 commented Oct 14, 2020

ohh oops, sorry. I thought it defaulted to squash and merge. I'll make sure to double check next time.

@chapulina
Copy link
Contributor

I thought it defaulted to squash and merge.

Yeah it annoyingly defaults to the last used, which may be "merge commit" or "rebase and merge" in case of ports across branches 😕

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏰 citadel Ignition Citadel enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants