World_vertex_coords flag required in canvas_item shaders

[fracteed]

Godot version:

3.0.3/3.0.4

OS/device including version:

Issue description:

After many attempts to get world space coordinates for the vertex and fragment shaders in a canvas item shader, I have come to the conclusion that it is not possible, without resorting to shader parameter hacks.(If I am wrong then please let me know!).

With spatial shaders it is very easy to do by turning on the render_mode "world_vertex_coords" and it works perfectly without having to play around with multiplying matrices.

Would be great to have this option for canvas_item shaders as well. After much googling and asking the community, there seems to be a lot of confusion on this matter and it is a much needed option.

Steps to reproduce:

• as can be seen in the demo scene attached, when moving the player (arrow keys), it gets red as it gets towards the top left corner of the screen. If you move the camera anywhere, the same thing happens. It should be getting red as it approaches the godot sprite which is at the world origin(0, 0). It is instead getting red at the screenspace (0, 0). I am using world_pos = (WORLD_MATRIX * vec4(VERTEX, 0.0, 1.0)).xy; to derive the world position.

Minimal reproduction project:

world_pos_canvas.zip

[jotson]

I don't fully understand matrix multiplication BUT... it looks like this is working except that world_pos isn't undergoing a projection transform. If you remove the camera or make it not current then you can see that your code works. It's just that the player is being transformed by the camera but world_pos is not.

It seems like you should be able to do something to world_pos with PROJECTION_MATRIX but I have no idea what. Someone with more math chops might be able to help.

An alternative is to just pass in world_pos as a uniform instead of calculating it.

varying vec2 world_pos;
uniform vec2 world_object_position;

void vertex() {
	world_pos = world_object_position + VERTEX;
}

...and in your _process() callback:

$player_sprite.material.set_shader_param('world_object_position', $player_sprite.global_position);

[jotson]

This almost works but it seems to ignore camera position and zoom:

void vertex() {
	world_pos = ((PROJECTION_MATRIX * WORLD_MATRIX) * vec4(VERTEX, 0.0, 1.0) * inverse(PROJECTION_MATRIX)).xy;
}

[fracteed]

thanks @jotson for trying to solve this! I also ended up trying very matrix combo I could think of, but couldn't solve it. I think it is because it is missing the camera and inverse camera matrices that the spatial shader has. My knowledge of matrix math is very limited, so I am not sure.

I also had tried feeding in the player position as a uniform. It does work , but it means you need a separate shader for each object that uses it, that is why I need a pure shader approach. I am trying to use it on my seaweed so that it would distort more and change colour as the player touched it.

Btw, I did try the same idea in 3d with spatial shader and it works easily since it has the world vertex coords flag builtin. Am hoping that a smart dev sees this issue and adds this to canvas shader so that we don't have to worry about all the matrix math ourselves :)

[clayjohn]

I started working on this today. It shouldn't be too big of a problem (famous last words). However, I ran into a bug (#23976) and ended up spending all my time on that.

I want to document a few things in case I don't end up finishing this and someone takes over:

• WORLD_MATRIX actually goes from image space to View/Camera space, not world space. (In GDscript it is the same as get_global_transform_with_canvas(), while what we want is just get_global_transform which takes the local coordinates and transforms them to world coordinates)

• In visual_server_canvas at line 160 the xform Transform2D is assigned to the canvas_item's final_transform which is then read in rasterizer_canvas_gles3 and used to set the MODELVIEW_MATRIX (named modelview_matrix in the shader, but accessible as WORLD_MATRIX to users).

• final_transform is calculated during rendering as the renderer works its way down the tree. It starts with the canvas transform and then multiples each branch as it works down. I think obtaining a proper world matrix is as simple as maintaining an transform beside that one that doesnt take into account the canvas transform and then passing that to the shader

[fracteed]

@clayjohn thanks so much for working on this!

[clayjohn]

Here is my WIP branch implementing world_vertex_coords in canvas_item shaders. It is currently working in gles3. It was even more simple than I thought. It should not take much to add it to the gles2 branch. I am just going to move the matrix calculations out of the drivers.

However, I am not sure that once I make a PR it will be merged into 3.1 as they have paused adding features, it may not even be merged for 3.2 as we work towards a new renderer.

Accordingly, a work around for those of you that really want the functionality is to pass the global transform from the node into the shader as a uniform (this doesn't work now due to a bug, but is fixed with #23976).

material.set_shader_param("global_transform", get_global_transform())

Then, in your vertex shader:

uniform mat4 global_transform;
varying vec2 world_pos;
void vertex(){
    world_pos = (global_transform * vec4(VERTEX, 0.0, 1.0)).xy;
}

[fracteed]

Thanks @clayjohn . Looks very promising, so I hope that at least that bugfix will be added to 3.1. It would explain the strange results I was getting when passing in shader parameters.

[bfishman]

Just wanted to thank @clayjohn and also to bump this issue. Not to sing a sob story but I just spent several hours bug chasing before realizing that WORLD_MATRIX doesn't properly account for camera transforms in canvas shaders. At least this issue was easy to find.

[clayjohn]

I'm sorry that happened to you! I also updated the docs to explain it as well http://docs.godotengine.org/en/latest/tutorials/shading/shading_reference/canvas_item_shader.html#vertex-built-ins.

[bluenote10]

@clayjohn Thanks as well for the work-around! A bit of a pity that this has been dropped from 3.2. Here's a slight extension of the work-around: What makes the work-around tedious at first is that the shader preview in the editor doesn't work, and that duplicating any object requires a "make unique" on the material. It looks like this can be automated by a small variation:

tool
extends Sprite

func _ready():
    print("Duplicating material")
    material = material.duplicate()

func _process(_delta):
    material.set_shader_param("global_transform", get_global_transform())

[Calinou]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!

[Toemsel]

Any status updates on this? It has been closed but no solution has been provided as far as I can see.

[RonTang]

Any status updates on this? It has been closed but no solution has been provided as far as I can see.

There is a ugly solution "Duplicating material" "Passing global transform every frame"

[Calinou]

There is a ugly solution "Duplicating material" "Passing global transform every frame"

Per-instance uniforms are coming to Godot 4.0 🙂

[RonTang]

There is a ugly solution "Duplicating material" "Passing global transform every frame"

Per-instance uniforms are coming to Godot 4.0 🙂

Very very good.Can not wait any longer. Thanks the Godot members~

[tripodsan]

sorry, I don't seem to get it work. I try to draw a SDF for a polygon, based on screen coordinates.
so I have a simple polygon with 4 points:

func _ready() -> void:
  print("Duplicating material")
  material = matFrame.duplicate()

func _process(delta) -> void:
  if is_inside_tree():
    var vx = [];
    for v in polygon:
      vx.append(get_global_transform().xform_inv(v));
    material.set_shader_param('v0', vx[0])
    material.set_shader_param('v1', vx[1])
    material.set_shader_param('v2', vx[2])
    material.set_shader_param('v3', vx[3])

and a canvas shader with:

uniform vec2 v0;
uniform vec2 v1;
uniform vec2 v2;
uniform vec2 v3;

float distToLine(vec2 p, vec2 q, vec2 a) {
  vec2 pq = q - p;
  vec2 n = normalize(vec2(pq.y, -pq.x));
  vec2 pa = a - p;
  return abs(dot(n, pa));
}

void fragment() {
  vec2 fx = FRAGCOORD.xy;
  float d = distToLine(v0, v1,  fx);
  d = min(d, distToLine(v1, v2, fx));
  d = min(d, distToLine(v2, v3, fx));
  d = min(d, distToLine(v3, v0, fx));
  COLOR.rgb = vec3( d / 250.0);
}

but somehow it's still dependent on the position in the screen:

https://i.imgur.com/ldVlPSV.mp4

any hints?

(apart from the fact that the SDF is actually unsigned and wrong :-)

[kartonrad]

oof this is still very annoying, as its not feasible to pass the Global transform to every tile in a tilemap

this makes it basically impossible to sample textures bigger than the individual tiles, based on world position.
so ugh it seems like i'll have to wait for 4.0 after all

it's fine, you guys are doing amazing work, i'm just malding yk

[DavidEichmann]

I came here looking for a way to get world position in the fragment function. I've found a simple solution here by using a varying. Note that this works because varying values are interpolated between vertexes:

shader_type spatial;
render_mode specular_schlick_ggx;

varying vec3 worldPos;

void vertex() {
	vec4 worldPos4 = WORLD_MATRIX * vec4(VERTEX, 1);
	worldPos = worldPos4.xyz / worldPos4.w;
}

void fragment() {
	// use `worldPos` here
}

[MonoGameDev]

oof this is still very annoying, as its not feasible to pass the Global transform to every tile in a tilemap

As far as I understand, you don't have to. Multiplying with WORLD_MATRIX already gives you the accurate position in relation to the viewports origin. Or in other words, in relation to the top left corner of your screen.
Only thing that's missing is the camera transformation. You have to pass that as a uniform, but it's generally the same matrix for every canvas_item in a viewport/canvas_layer, so there is no need to pass every individual, global transform of each tile. Just pass the affine inversed canvas transform to the entire tilemap.

Here is how to pass the correct matrix:

const MATERIAL = preload("res://my_shader_v1.0/material.tres")


func _process(delta):
	MATERIAL.set_shader_param(
		"camera_matrix",
		get_viewport().canvas_transform.affine_inverse()
	)

Here how the transformation in the shader should look like:

void vertex() {
	world_coord = (
		camera_matrix * WORLD_MATRIX * vec4(VERTEX, 1.0, 1.0)
	).xy;
}

Note, that I declared the camera_matrix uniform as a mat4
uniform mat4 camera_matrix = mat4(0.0);

I think mat3 should work as well if you restructure the multiplication a bit, but I had trouble with it and was too lazy to investigate that any further.

[2fd5]

Sorry to resurrect the old issue that is closed, I am unsure how global uniform in Godot 4 actually resolves the issue here with the camera. Or is it any other feature added that helps with the issue here. Are there any examples in the documentation on how to do that ? I could not find any.
The post mentioned here https://godotengine.org/article/godot-40-gets-global-and-instance-shader-uniforms/ does mention the use cases in Zelda, but does not show how one would go about doing that in Godot.
Are we still missing come built-ins for canvas_item shader ? is there a need for a new proposal. Or is there just unconvenient way of doing it ?
This is highly technical issue that users might need some landmarks to navigate, is godot-proposals a good place for proposing documentation that is lacking ?

[akien-mga]

Implemented by #81160.