5 WESL Proposals #20

ncthbrt · 2024-09-05T15:52:59Z

This draft PR includes several proposals that build upon one another.
In order these are:

Import - Simplification of imports
Modules - Encapsulation and namespacing of elements
Module Signatures - Visibility & Typechecking
Include - Inheritance/Extensibility
Module Generics - Abstraction & Reusability

…yet complete

stefnotch · 2024-09-06T14:55:08Z

Modules.md

+
+This scheme means that if a given symbol has been included into the global scope, it is not mangled at all. It has the absolute minimum number of parts in the path!
+
+An example of how to apply the algorithm described above, would be the name used in mangling the `quat_from_euler` function. The name chosen to be mangled would be `Math::Float::quat_from_euler`, rather than `Math::FloatMath::quat_from_euler`. This is because while they have the same number of parts in their paths, `Math::Float::quat_from_euler` has fewer number of characters.


If I understand this correctly, this means that the host code (Rust or Typescript) doesn't know what the mangled name is until after the modules have been compiled. Is that correct?

With conditional compilation, the mangled name could also unexpectedly change.

Hmmm. Interesting point. My motivation was originally to reduce the size of the code using this approach and resolve to the most public facing name. Most modules outside of the global scope would contain implementation details and not be for public use anyway.

However bindings are a bit of an exception as users may still want to be able to introspect on them and to also generate binding code.

So a dilemma!

Regarding "generate binding code", would it be reasonable to say that that is part of what a good linker should offer? Or rather, a good linker should offer some language specific APIs for it.

As in, we aren't expecting people to generate perfect bindings from generated code. Generated code could, for example, no longer have comments. Good bindings code would include documentation comments.

So for the generated output, we only need something that is predictable and usable. (compared to raw WGSL)

That makes sense. Do you have a suggestion for achieving this? Would prefer if we didn't have to special case the output names of entry points (which was another reason for choosing this scheme) though understand it might be necessary.

mighdoll · 2024-09-09T03:56:28Z

GenericModules.md

+}
+
+// Abstract representation of a binary operation. 
+mod sig BinaryOp<OpElem: Type, LoadElem: Type> {


Do you think generics w/o type bounds are a useful intermediate step?
I'm not certain, but I'd been hoping so to make things easier.
(w/o type constraints generics would be more about getting the flexibility to replace the string templating that current wgsl users do. But that weaker form wouldn't add type safety, so some errors would be caught later perhaps by the dawn/wgpu parser)

It might be but would entail a breaking change once support for constraints were added.

why necessarily a breaking change? I'd think simple substitutions e.g. f32 for LoadElem would be sound even when better typechecking comes in later. Inference would reduce boilerplate, but not render the boilerplate unsound.. hmm.. I bet there's a case I'm missing, can you explain?

@mighdoll The problem is that this design does not allow you to access any members of the module were the generic constraint omitted (which is why it's not permitted to omit the constraint in the grammar in the first place). So if you added typechecking after the fact, you'd get a lot of errors.

This is because this design for simplicity does not use type inference. Only checking.

Let me restate to see if I understand:
. if we were to release an edition with a generics-light (no contraints, no typechecking, no inference)
. it's perhaps possible to add typechecking in a later edition in principle, but
. if the next stage is generics-checked (constraints, typechecking, no inference)
. generics-checked will want to require explicit constraints for every type parameter (because it doesn't have inference)
. so best not to allow unconstrained type parameters, lest generics-light code become incompatible with generics-checked

As mentioned on discord, there is one way to preserve compatibility in this design which is to add the concept of an "any" module type, which much like in typescript would mean that module arguments that do not specify type constraints would be unchecked

mighdoll · 2024-09-09T04:10:45Z

Include.md

+// Tells the linker to resolve calls to `resolve_pbr_inputs`
+// to this function instead of the base. This is constrained to the 
+// current namespace; in this case the namespace is the global scope.
+patch fn resolve_pbr_inputs(pbr_inputs: &PbrInputs) {


Patch seems powerful! too powerful?
Are patch targets constrained somehow? (I'm thinking about the library case, where a library author wouldn't want messy internals to become public api because a user can patch them.)

The visibility mechanism in this proposal would be the only way to constrain patch targets which is quite limiting I agree. An alternative idea would be something like a "virtual" keyword.

mighdoll · 2024-09-09T04:15:23Z

Modules.md

+
+Currently all symbols in wgsl share a single global namespace with a prohibition against symbols with the same name. Many existing implementations of imports (not unlike [our proposal](./Imports.md)) simply add to this global scope. For relatively self contained projects this is not a problem, however when considering a broader ecosystem of packages containing reusable shaders, collisions become much more likely. 
+
+Additionally WGSL provides very little way to encapsulate and organise code. This proposal would pave the way for both


I think this is a keen observation. there's just no current way to describe a group of elements in wgsl right now!

mighdoll · 2024-09-09T04:25:44Z

Modules.md

+Currently all symbols in wgsl share a single global namespace with a prohibition against symbols with the same name. Many existing implementations of imports (not unlike [our proposal](./Imports.md)) simply add to this global scope. For relatively self contained projects this is not a problem, however when considering a broader ecosystem of packages containing reusable shaders, collisions become much more likely. 
+
+Additionally WGSL provides very little way to encapsulate and organise code. This proposal would pave the way for both
+[Module Interfaces](./ModulesInterfaces.md) and [Generic Modules](./GenericModules.md). The former would allow a graphics programmer to optionally control the visibility and typecheck the symbols within a module, while the latter would allow for more reusable code and the ability to build powerful abstractions. 


It's impressive how you show the mod scaffolding can be stretched to cover so many features.
What's the +/- for building those features as 'mod' extensions vs building them separately?

While hard to list the pros and cons without having something concrete to compare it against, there are a few things I'd call out.

On the plus side, the contents of modules are valid wgsl; there are no additional constructs to transpile other than mangling the symbols within a module.

On the minus is that sometimes it may be more verbose than e.g. generic functions — this could be solved by syntactic sugar that coerces functions and constants into modules.

Another minus is that this style of programming may be unfamiliar to many graphics programmers. Luckily this is somewhat ameliorated by having few additional concepts in the WESL language to learn other than modules.

mighdoll · 2024-09-09T04:31:25Z

This is a really extensive proposal! It'd take us quite a while to work through all of it, figure out any potential issues, improve parts, etc. Can you see a way to split it up into smaller pieces?

ncthbrt · 2024-09-09T06:37:46Z

This is a really extensive proposal! It'd take us quite a while to work through all of it, figure out any potential issues, improve parts, etc. Can you see a way to split it up into smaller pieces?

@mighdoll

For bevy PBR stuff, think import, modules and include would be enough.

Typechecking/module signatures could probably be separate.

And generics could be a dependency on one or both.

mighdoll · 2024-09-09T16:40:08Z

For bevy PBR stuff, think import, modules and include would be enough.

Oh! I would have guessed patch (+ import of course). Can you explain in a toy example?

(I guessed that the key PBR problem was enabling a library to provide a large set of extension points in a deep chain of library functions, and that patch lets the library user replace any one of those functions, efficiently turning every function into an extension point.)

ncthbrt · 2024-09-09T17:00:05Z

@mighdoll you're right patch needs to be in that too.

ncthbrt added 3 commits September 5, 2024 13:59

Finished initial doc for alternative import specification.

20735c5

Added module proposal

9d467dd

Added a draft of module interfaces. However the type checking is not …

cebdb85

…yet complete

ncthbrt marked this pull request as draft September 5, 2024 15:53

ncthbrt requested review from stefnotch, mighdoll and k2d222 September 5, 2024 16:01

ncthbrt and others added 3 commits September 5, 2024 22:36

Update ModulesInterfaces.md to add section on type checking roles

596b494

Updated proposal to reuse aliases for module aliases

8b3bbd0

Added Include specification

10966a2

stefnotch reviewed Sep 6, 2024

View reviewed changes

Started working on specification for generic modules

2936a75

mighdoll reviewed Sep 9, 2024

View reviewed changes

Finished first draft of GenericModules

849d7f8

ncthbrt marked this pull request as ready for review September 9, 2024 12:16

ncthbrt self-assigned this Sep 9, 2024

ncthbrt changed the title ~~Various related proposals~~ 5 WESL Proposals Sep 9, 2024

ncthbrt and others added 3 commits September 9, 2024 15:37

Delete GenericFunctions.md

44c2913

Added original wgsl syntax.bnf file

f5565a1

Added attributes to load statement

37ec96c

mighdoll mentioned this pull request Sep 15, 2024

Split Import Spec: Source code paths vs module paths? #24

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

5 WESL Proposals #20

5 WESL Proposals #20

ncthbrt commented Sep 5, 2024 •

edited

Loading

stefnotch Sep 6, 2024

ncthbrt Sep 6, 2024 •

edited

Loading

stefnotch Sep 6, 2024

ncthbrt Sep 6, 2024 •

edited

Loading

mighdoll Sep 9, 2024

ncthbrt Sep 9, 2024

mighdoll Sep 9, 2024

ncthbrt Sep 9, 2024

ncthbrt Sep 9, 2024 •

edited

Loading

mighdoll Sep 9, 2024

ncthbrt Sep 9, 2024

mighdoll Sep 9, 2024

ncthbrt Sep 9, 2024

mighdoll Sep 9, 2024

mighdoll Sep 9, 2024

ncthbrt Sep 9, 2024 •

edited

Loading

mighdoll commented Sep 9, 2024

ncthbrt commented Sep 9, 2024

mighdoll commented Sep 9, 2024

ncthbrt commented Sep 9, 2024


		This scheme means that if a given symbol has been included into the global scope, it is not mangled at all. It has the absolute minimum number of parts in the path!

		An example of how to apply the algorithm described above, would be the name used in mangling the `quat_from_euler` function. The name chosen to be mangled would be `Math::Float::quat_from_euler`, rather than `Math::FloatMath::quat_from_euler`. This is because while they have the same number of parts in their paths, `Math::Float::quat_from_euler` has fewer number of characters.


		Currently all symbols in wgsl share a single global namespace with a prohibition against symbols with the same name. Many existing implementations of imports (not unlike [our proposal](./Imports.md)) simply add to this global scope. For relatively self contained projects this is not a problem, however when considering a broader ecosystem of packages containing reusable shaders, collisions become much more likely.

		Additionally WGSL provides very little way to encapsulate and organise code. This proposal would pave the way for both

5 WESL Proposals #20

Are you sure you want to change the base?

5 WESL Proposals #20

Conversation

ncthbrt commented Sep 5, 2024 • edited Loading

Choose a reason for hiding this comment

ncthbrt Sep 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncthbrt Sep 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncthbrt Sep 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ncthbrt Sep 9, 2024 • edited Loading

Choose a reason for hiding this comment

mighdoll commented Sep 9, 2024

ncthbrt commented Sep 9, 2024

mighdoll commented Sep 9, 2024

ncthbrt commented Sep 9, 2024

ncthbrt commented Sep 5, 2024 •

edited

Loading

ncthbrt Sep 6, 2024 •

edited

Loading

ncthbrt Sep 6, 2024 •

edited

Loading

ncthbrt Sep 9, 2024 •

edited

Loading

ncthbrt Sep 9, 2024 •

edited

Loading