Inconsistent behavior with ModelGraphics.nodeTransformations

[richard3d]

I have an entity that I have defined along with a glTF model. There is a single animation that exists in the file but I disable it by default (runAnimations: false). I have noticed an odd behavior in particular when translating nodes. If I translate a node that has no rotation animation keys defined, the node translates as expected relative to the parent node's local coordinate system i.e. a translation of 10,0,0 results in the node being moved ten units along the parent node's x-axis. However, when translating a node that has rotation keys defined, the transformation moves the node along its own x-axis rather than the parent's axis as expected. Again, the animation is totally disabled so its not like the timeline is overwriting the value or anything like that. I created a post in the mailing list asking about this behavior but have not received any responses. You can view the full history of the discovery here

[richard3d]

Currently we have found work around to this problem in how models are rigged before exporting to glTF. While the inconsistent behavior is still a bit of a headache I think this can be resolved through better documentation and/or guidelines when bringing glTF models into Cesium.

[hpinkos]

@lilleyse thoughts?

[OmarShehata]

@richard3d I know you mentioned you found a workaround in the forum thread as well, but I don't think you ever described what it was. Can you explain what your workaround was when rigging the model before exporting it to Cesium? It'll give us some more insight into what I believe is a bug in resolving the orientation of child/parents during animations.

And just to close the loop, for anyone else running into similar problems, Richard has made a small library on top of Cesium for giving you greater control over animations outside the timeline:

Forum thread: https://groups.google.com/d/msg/cesium-dev/wOXc8MQL40A/An9TnWeBAQAJ
Repo: https://github.com/ProminentEdge/Cesium-ModelAnimationPlayer

It seems to work pretty well from my tests!

[richard3d]

Hey everyone sorry for the late response but I wanted to run some more tests to give the best information possible. After working with @OmarShehata and analyzing some other gltf models, I have come to the conclusion that I mistakenly described the problem due to the original model I was working with. It seems the TRS behavior of nodeTransformations is indeed consistent (always translates relative to a given node's own local coordinate system, but expressed from its initial TRS value). The work around for us was to ensure that a node's local transform was oriented the way we wanted before exporting (e.g. orienting local transform of a model/node in 3ds Max). After we identified the behavior I was able to adjust the code for the animation player I was working on to get it to work correctly. There are some improvements/considerations I would offer though:

• Debugging models, animations, and node transformations in Cesium is VERY difficult at the moment, part of this is due to the fact that specifying nodeTransformations in Cesium is different than how they are expressed in glTF (which is always relative to a parent). If you look at the code for the animation player you will see all the places I have to transform animation values expressed in parent node space to get them into the animated node's own local space. Consider expressing them the same as glTF (preferred option) or provide clearer documentation.

• I know it increases the memory footprint but I think it would be worth at least storing a pointer/name to the parent node in NodeTransformationProperty. At the very least it makes it easier to calculate and debug worldspace transforms of nodes. Otherwise one must have access to AND parse the original glTF to get at this information.

I am curious to hear any feedback on these suggestions. I will close this issue shortly (since it is misleading/not relevant at this point) and create other ones for the improvements listed. You can see a full demo of my current work here: https://cesium-modelanimationplayer.glitch.me/
and source for the demo project here: https://glitch.com/~cesium-modelanimationplayer .
It illustrates timeline independent animation along with ability to debug transforms of nodes.

[cesium-concierge]

Congratulations on closing the issue! I found these Cesium forum links in the comments above:

https://groups.google.com/forum/?hl=en#!topic/cesium-dev/-3uqvsuF06I
https://groups.google.com/d/msg/cesium-dev/wOXc8MQL40A/An9TnWeBAQAJ

If this issue affects any of these threads, please post a comment like the following:

The issue at #7615 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

[lilleyse]

Debugging models, animations, and node transformations in Cesium is VERY difficult at the moment, part of this is due to the fact that specifying nodeTransformations in Cesium is different than how they are expressed in glTF (which is always relative to a parent).

Just to make sure I understand the problem I drew a picture of what the different transforms look like.

Is it that you expect the second image while Cesium does the third?

My interpretation of the glTF spec is that transforms are always in a local coordinate system, not relative to their parent:

https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#transformations

Any node can define a local space transformation either by supplying a matrix property, or any of translation, rotation, and scale properties (also known as TRS properties).

I'd be curious in seeing what ThreeJS or BablyonJS do here.

[richard3d]

I think there needs to be some more clarifying information before I answer:

TRS properties are converted to matrices and postmultiplied in the T * R * S order to compose the transformation matrix; first the scale is applied to the vertices, then the rotation, and then the translation.

Based on this info and assuming the parent box is a unit cube and has axes +y running up vertically with your drawing, +x to the right..etc. I would interpret

• figure 1's triangle as having a Trans = (0,1,0), Rot = (0,0,0)
• figure 2 Trans = (1,0,0), Rot = (0,0,90) <- might be negative due to handedness but I'm ignoring for simplicity
• figure 3 Trans = (0,1,0), Rot = (0,0,90) <- same

The BabylonJS Sandbox is what originally lead me to these conclusions. If you load this file (.glb in the zip) in the editor and rotate the node mixamorig:LeftUpLeg_055. You will see that the rotation occurs in the frame of the parent node's axes. Setting the translation value to 0,0,0 also sets the node position to the same location as the parent. Likewise setting the x component translation value will move the node relative to the parent's x-axis not according to the node's own axis. I do realize this may just be their implementation or interpretation.

Again, Cesium does not necessarily perform node transformations (in ModelGraphics) incorrectly, it just does it in a different space i.e. relative to the node's original TRS than I expected. If you think I am also mis-interpreting the spec please let me know!

.

[emackey]

Hi guys. I don't think the glTF spec says anything about clients' application of their own node transforms to the model, be they replacements or in addition to existing node transforms.

But I do want to point out the "original matrix" logic will change slightly in a PR I currently have open, #7835. Prior to this PR, the ModelVisualizer thought it was the only client modifying the matrix, so it made itself responsible for keeping a hash of "original" matrices, which are just in whatever state the ModelVisualizer happens to find them in the first time it goes to update one of them.

In this PR, the storing of "originals" changes to be on the publicNode itself, before anything else has a chance to modify them. This is because a new feature (articulations) needs access to the originals as well, and could potentially alter these node matrices before ModelVisualizer or anyone else gets a peek at them. So the individual nodes are now responsible for storing their own "original" state, as specified in the glTF file prior to any animations, nodeTransformations, or articulations being applied.

I don't know if this will have any implications for your use here. Hopefully not, but it might make the originalMatrix more immune to changes from model animations.

[richard3d]

@emackey I agree with what you said regarding the spec:

I don't think the glTF spec says anything about clients' application of their own node transforms to the model, be they replacements or in addition to existing node transforms.

I don't think your work will affect my usage currently. This issue came about due to my assumption that node transformations for ModelGraphics would behave the same (TRS being performed in the same space / reference axes) as glTF spec would expect for TRS values specified for a node in a file. Hence why I closed the issue originally. I think @lilleyse is examining some of my other comments from the issue currently to see if there are any improvements to be made and for correctness.

[lilleyse]

Based on this info and assuming the parent box is a unit cube and has axes +y running up vertically with your drawing, +x to the right..etc. I would interpret

• figure 1's triangle as having a Trans = (0,1,0), Rot = (0,0,0)
• figure 2 Trans = (1,0,0), Rot = (0,0,90) <- might be negative due to handedness but I'm ignoring for simplicity
• figure 3 Trans = (0,1,0), Rot = (0,0,90) <- same

Okay I think we are in agreement then. If someone sets these values to ModelNode.matrix directly it will result in the images I drew. I was interpreting "relative to parent axis" in a different way but now I understand what you mean.

I have nothing else to add then, except that in the future we may to consider changing ModelGraphics node transforms to allow for full control over the node's matrix rather than just acting as an offset - to be consistent with the behavior already in ModelNode.

[emackey]

in the future we may to consider changing ModelGraphics node transforms to allow for full control over the node's matrix

I'm not totally on board with the need for that. Obviously we can't break existing use of nodeTransforms. But aside from the compatibility concerns, I think it makes some sense to transform a node in its own coordinate system. If you really need the parent's system, just remove the static transform from the glTF file, and apply that as the starting nodeTransform instead.

Also, now that articulations are starting to join this party, I'm hoping to encourage users to ramp up on that instead of nodeTransforms. We don't have it in CZML yet, but that's coming. Articulations can automate certain tasks, like spinning all wheels at once, or causing two halves of a payload fairing to separate, or multiple segments of a solar panel to deploy at the same time, etc. A certain commercial launch vendor (not sure if I'm allowed to name them here) has been using articulations driven by realtime telemetry from launches for many, many years, in STK. Articulations are applied in the node's local system, after any model-supplied static node transforms.

[lilleyse]

I'm not totally on board with the need for that. Obviously we can't break existing use of nodeTransforms. But aside from the compatibility concerns, I think it makes some sense to transform a node in its own coordinate system. If you really need the parent's system, just remove the static transform from the glTF file, and apply that as the starting nodeTransform instead.

Depending on how comfortable we are with bridging the entity and primitive APIs, another option would be to add a ModelNode getter to the entity API so that users that wish to change the underlying matrix may do so. #7761 brings up a similar use case.