Make physics layers and masks logic simple and consistent

[pouleyKetchoupp]

Describe the project you are working on

Godot Physics

Describe the problem or limitation you are having in your project

The current logic for layers and masks can be confusing.

The main definition of layers and masks states:

collision_layer
This describes the layers that the object appears in. By default, all bodies are on layer 1.

collision_mask
This describes what layers the body will scan for collisions. If an object isn't in one of the mask layers, the body will ignore it. By default, all bodies scan layer 1.

Source: https://docs.godotengine.org/en/latest/tutorials/physics/physics_introduction.html#collision-layers-and-masks

But the logic actually applied in all cases is:
A contact is detected if object A is in any of the layers that object B scans, or object B is in any layer scanned by object A.

Source: https://docs.godotengine.org/en/stable/classes/class_physicsbody.html#class-physicsbody-property-collision-layer

This makes objects setup cumbersome when it comes to certain interactions, because you always need to make sure layers are properly set both ways when you want to avoid interactions between certain objects.

Some cases are also not possible at all, for example:

• Areas that can detect a certain layer without being detected by objects from this layer.
• Kinematic bodies that can interact with other kinematic bodies using move_and_collide or move_and_slide without having interactions the other way around.
• Advanced interactions between rigid bodies.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The proposed solution is to change the logic to the following in all cases:
Object A can detect a contact with object B only if object B is in any of the layers that object A scans.

This statement would have to be consistent in all possible situations to help with documentation and make the concept easy to understand without edge cases to learn.

This is how it would work for different scenarios:

• Area detection

An area can detect any object which is in the collision_mask of the area, whatever the value of collision_layer for the area.
Area A can be detected by area B only if area B's collision_mask contains one of the layers from area A's collision_layer.

• Kinematic body motion

When using move_and_collide or move_and_slide, a kinematic body can detect collision only with objects if the kinematic body's collision_mask contains one of the layers from the object's collision_layer.

That means it's possible for detection to be non-symmetrical, which can be useful in certain cases.

Example:
A player KinematicBody needs to push an enemy's KinematicBody around, and both can push rigid bodies.
Player: collision_layer=1, collision_mask=1
Enemy: collision_layer=2, collision_mask=1
Rigid bodies: collision_layer=3, collision_mask=1+2+3
Floor: collision_layer=1, collision_mask=1+2+3
This way, the player doesn't detect enemies and will just move without being stopped (it can overlap by moving through).
The enemy will detect the player so it will collide and stop (it will be pushed out when overlapping).
Both the player and enemy don't detect rigid bodies and can push them around (they will be pushed out when overlapping).

• Rigid body interaction

In order to make things consistent, rigid body collision detection would also support asymmetrical settings. When a pair of rigid bodies is detected, rigid body A can collide with rigid body B only if body A's collision_mask contains one of the layers from body B's collision_layer.

In case body A collides with body B, but body B doesn't collide with body A, body B will simply move without getting any impulse, and body A will treat body B like a kinematic body (body A will be entirely pushed out from the contact).

This will allow advanced interactions, where rigid bodies can be treated as kinematic bodies only in certain pairs, while using all the types of interactions rigid bodies have (gravity, forces, impulses, contacts,...).

Example:
A bowling ball needs to push pins without being affected at all, while rolling and hitting walls.
Ball: rigid body with collision_layer=1, collision_mask=1
Pin: rigid body with collision_layer=2, collision_mask=1+2
Floor and walls: static body with collision_layer=1, collision_mask=1+2
This way the ball will hit pins and push them around without losing velocity, while pins will collide with the ball and with each other.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Some PRs have already been made to change certain cases:
-Area detection: GH-42268
-Kinematic body motion: GH-42641

The remaining things to do are:
-Rigid body interaction: handle special cases when one of the bodies doesn't detect the other one
-Documentation and tutorials need to be updated

If this enhancement will not be used often, can it be worked around with a few lines of script?

Some scenarios can be done only with complex workarounds.

Is there a reason why this should be core and not an add-on in the asset library?

It has to be part of the logic inside of the physics simulation.

[NathanLovato]

Yes, please! A much needed simplification.

I can confirm from years making Godot tutorials that the current way it works confuses people, and that they would expect it to work exactly how you intend to change it.

[farfalk]

Please yes. My brain can't help to think of layers and mask in the asymmetrical way you're proposing, and I often end up crashing against the truth thinking I'm experiencing a bug.

I'm actually happy to see this proposal, I felt quite dumb about this until now!

[YuriSizov]

@seppoday Please don't bump issues to just show your support, use the thumbs up reaction on the first post to do so. Reserve comments for actual discussion, suggestions and questions.

[Yogoda]

Also suggests renaming "mask" with "interacts with". Mask is a technical term related to the internals and makes it harder to understand for people not used to using bitmasks (most people).

[2plus2makes5]

"Some cases are also not possible at all, for example:

Areas that can detect a certain layer without being detected by objects from this layer.
Kinematic bodies that can interact with other kinematic bodies using move_and_collide or move_and_slide without having interactions the other way around."

Maybe i didn't understand the issue but it seems to me that those things can be done:
1)if there are two layers "area" and "player" you can make the player scan no layers while an area can scan the "player" layer.
2)you can have "player" and "enemies" layers and make the player scan the "enemies" layer while enemies don't scan the "player" layer.

[pouleyKetchoupp]

@2plus2makes5 It's difficult to do this kind of setup right now, because areas are checking masks both ways. There are old reported issues about this, like godotengine/godot#7644.

[TackerTacker]

It all sounds good to me in the proposal, but are you sure you thought of everything?
I'm always skeptical of changes like this, where it's a well known convention that basically every engine is using.
I ask myself, how likely is it that all these people that have worked on all these engines really just follow blindly what others did, or is it more likely that this is the best solution/compromise that we've found through trial and error?
Like for example you list things that aren't possible with the current system and that they are possible with the new one which is great, but did you also think through all the cases that are possible with the current system and if they'll still be possible with the new one, and/or still as easy to setup as before?

I agree that the current system is not easy to understand, but I think there could be less drastic changes that could help with that. For example Collision Layer and Collision Mask are not great names in terms of comprehensibility, something like
"Collision layer" becomes "My collision"
"Collision mask" becomes "Collisions I detect"
would help a lot. Is it elegant, no. Is it easier to understand, absolutely IMO

I'm not an expert though, I just thought I share my concerns because this sounds like a pretty big change.

[pouleyKetchoupp]

@TackerTacker It is a pretty big change and I can't personally guarantee that I thought of every single possible case, that's why I'm going through this proposal.

The idea is to have this discussion early enough to implement the change before 4.0 alpha is out, which will leave a lot of extra time to find possible edge cases, and if it happens decide how to make the proper fix. If something goes wrong there's still the option to revert the change, but I doubt that will happen.

[2plus2makes5]

@pouleyKetchoupp you are right, Areas without masks also detect collisions! I have never noticed this, but with the right monitoring and monitorable flags the problem is solved, it's not an excuse to not solve the issue obviously, but in case you need a quick fix...

[viksl]

Does it mean this case: https://godotengine.org/qa/107079/area_entered-signal-called-areas-which-collisions-please would be actually dealt with, please?

I hope this proposal gets through since it doesn't block the earlier functionality but expands the current one in a straightforward and clear manner. Personally i see it as a plus.

@2plus2makes5 you can't solve the problem or all problems with the monitorable and monitoring flags unfortunately, there are pretty common cases which actually can't do it like that and you need extra code in collision to avoid these for every object which for 1 on 1 collision is just a one line but for 5 different objects that's either a special group created and checked or a switch/if statement with all the cases, ...

[AttackButton]

Yes, please! A much needed simplification.

I can confirm from years making Godot tutorials that the current way it works confuses people, and that they would expect it to work exactly how you intend to change it.

I'm testing godot 4 on my project and although at first it seems like a simplification, it didn't turn out to be the case.

If you have dozens of objects that need to collide in two ways (like godot 3.x) with a single object, instead of just setting the masks once on this object, you will need to set the layer of it in the masks of each of the other objects too.

So imagine after doing all the configuration above, you change your mind and don't want these objects to collide with this single object anymore, instead of just changing the collision mask on a single object, you will have to change the masks on each of the other objects again.

I know godot does not need to do everything like other engines, but it's like @TackerTacker said on his comment, this is the standard on every engine, why change that?

I really think situations like the one I described are why other engines use this convention.

*edit:
An example:

Using Godot 3.x layers and masks:

Car
layer   1
mask    2 3 4

Ambulance
layer   2
mask    4

Tractor
layer   3
mask    -


PoliceCar
layer   4
mask    3

To do the same thing using this proposal:

Car
layer   1
mask    2 3 4

Ambulance
layer   2
mask    1 4

Tractor
layer   3
mask    1 4


PoliceCar
layer   4
mask    1 2 3

[AttackButton]

@pouleyKetchoupp , could you reopen this proposal for discussion? Because I believe that the real effects of this change may take a while to be noticed, and may even occur only after the release of version 4.0.

This change the layer/masks design a lot for the users and does not simplify as intended. The CharacterBody infinite inertia needs to be an exception that is not to be resolved by the layers/masks design by default.

I really prefer to bring the infinity inertia property back.

[Calinou]

@pouleyKetchoupp , could you reopen this proposal for discussion? Because I believe that the real effects of this change may take a while to be noticed, and may even occur only after the release of version 4.0.

This change the layer/masks design a lot for the users and does not simplify as intended. The CharacterBody infinite inertia needs to be an exception that is not to be resolved by the layers/masks design by default.

I really prefer to bring the infinity inertia property back.

This proposal won't be reopened, as it has been addressed already. Please read godotengine/godot#31981 (comment) first, and continue the discussion in godotengine/godot#31981.

[AttackButton]

@pouleyKetchoupp , could you reopen this proposal for discussion? Because I believe that the real effects of this change may take a while to be noticed, and may even occur only after the release of version 4.0.
This change the layer/masks design a lot for the users and does not simplify as intended. The CharacterBody infinite inertia needs to be an exception that is not to be resolved by the layers/masks design by default.
I really prefer to bring the infinity inertia property back.

This proposal won't be reopened, as it has been addressed already. Please read godotengine/godot#31981 (comment) first, and continue the discussion in godotengine/godot#31981.

Ok. But why close this without the proper consensus? Likes are not consensus.

[Zireael07]

@AttackButton: 51 likes and 11 hearts is a pretty overwhelming vote in favor of this proposal. (I can barely recall a proposal which has more), therefore it was implemented (and closed)

The only critical voices started to appear AFTER it was implemented... which kinda makes sense as it's something that's difficult to predict ALL effects of...

[AttackButton]

@AttackButton: 51 likes and 11 hearts is a pretty overwhelming vote in favor of this proposal. (I can barely recall a proposal which has more), therefore it was implemented (and closed)

The only critical voices started to appear AFTER it was implemented... which kinda makes sense as it's something that's difficult to predict ALL effects of...

That's because I did test the pr, I even gave examples two or three comments above.

I was also one of the likes for the title statement, but after the proper testing, I came to the conclusion that this is not what is happening. That's why I'm saying that likes are different from the proper consensus, for the proper consensus more people need to test and give their opinions. A consensus based only on likes are opinions without the proper experimentation.

I did learn to use the new logic without any problems. I'm just saying it's not simplifying the old one as the title says.

One thing, I'm not saying anything personally, I'd like you to read my examples and arguments and test it yourself. I swear I'm not nitpicking, godot 4 will be great.

I really wish this pr could simplify the layer/masks design, but what's happening is the opposite. But if it's kept, I won't be angry, I'll use it without any problems.

** I'm not saying that every pr needs consensus, I'm saying maybe this one does, because it is a big change.
** It's not a matter of lack of documentation, but of bad design.
** Probably setting the layers and masks for a two way collision (like CharacterBody2D vs CharacterBody2D) via script gonna be the worst case. One will need to set the layers and masks of one object and need to find the other related objects as well.

[golddotasksquestions]

@AttackButton

I'd like you to read my examples and arguments and test it yourself.

I did that.

It seems to me you have a different expectation on what "simplification" would mean in the context of this PR.

To me (and it seems to OP and many others in this thread) the simplification is in clarity about what is happening.
If I understood your example and concerns correctly, the simplification you seem to think of seem to be about having doing less "work" of checking or unchecking collision layer and mask flags. Only uncheck one box instead of many, at the cost of clarity.

Both are fair ways to think about this problem, I think. Personally I feel like clarity is more important than less work in this case, and it seems a lot of other people think so too. Collision layer and masks can be a complex problem to teach and understand, as I have tried countless times on community channels. We need these rules to be simple imho, as in simple to explain and teach, even if this means having to check or uncheck more boxes.

Another substantial argument for favoring clarity (this proposal) over less work (3.X), is how this proposal makes the relationships much faster to grasp at a glance. Your comparison example above makes this immediately apparent I think. This is even more true, if you have a project you have not worked on in a while: one look at the Collision flags of a node and you know immediately how this nodes collision is going to behave. With the 3.X approach you would never know. You need to also look at all the other nodes collision setting and draw up a diagram for all the nodes to come to a conclusion how a single node is going to behave.