Interactions between JS objects and 'dynamic'

[kg]

Right now dynamic is a back door that essentially lets you use JS objects directly. If we implement proper DLR support, that means this back door goes away and becomes a pain to use JS objects.

In theory we could make DLR machinery properly interact with JS objects, but the generated code will still be riddled with callsite objects and such. Ideally we want a way to preserve the codegen for existing uses of dynamic on JS objects while still handling all the appropriate DLR use cases.

I'm not sure how to balance these two objectives. Opt-in/opt-out for full DLR support or for JS object support, so that the code generator knows what kind of JS to generate? Some sort of per-value annotation like packed structs, where a 'dynamic'/'object' value is tagged as being a 'JS Object' and any operations on it bypass the DLR? Not really sure. Thoughts and suggestions appreciated.

[iskiselev]

I'm not sure, if we really need support clear JS interop (current implementation) through dynamic.
If we want work with JS - we have Verbatium for it (may be we should provide some more helpers for it).
I thought, that we could introduce some helper, that will work same as Verbatium with API like:

public abstract class JsObject
{
    public abstract JsObject this[string name] { get; set; }

    public abstract JsObject Get(string name);
    public abstract T Get<T>(string name);
    public abstract void Set(string name, object value);

    public abstract JsObject Call(string name, params object[] args);
    public abstract T Call<T>(string name, params object[] args);
    public abstract JsObject Create(string name, params object[] args);

    public abstract T Cast<T>();
}

public abstract class JsFunction : JsObject
{
    public abstract JsObject Call(params object[] args);
    public abstract T Call<T>(params object[] args);
    public abstract JsObject Create(params object[] args);
}

We may add operation overload to this list.

At the same time, it is good idea to preserve working with JS through dynamic - but it will be details of Microsoft.CSharp.RuntimeBinder.Binder (and same binders for other languages) on JS side.

[kg]

Being able to use a DOM or JS API like this:
https://github.com/sq/JSIL/blob/master/Examples/WebGL/Page.cs#L129
Is pretty important. It's also important that the code maps naturally to JS so there aren't weird interop issues. This is one area where JSIL demolishes competing compilers because you can write natural code that consumes JS APIs. If we drop this and require people to use something like that hypothetical JsObject, that's a real step backwards. So many people have to manually build a bunch of statically-typed bindings for libraries like jQuery to use with other compilers, and that's a huge pile of error-prone busywork.

It may be necessary, though. :-(

[kg]

Thinking about this a bit more, we could probably lean on the same tricks packed arrays use. You can annotate a given dynamic field or property with some sort of [JSInterface] attribute, at which point the compiler will carry around special type information with values that enter/leave the field. The builtins like Builtins.Global will have that attribute, so any JS object you pull out of there gets the old dynamic codegen, and that allows the type info to reach local variables as well.

That'd probably cover most of these use cases, and then if necessary we can provide a DLR binder that does the rest.

[iskiselev]

It is only my opinion, but I really see no difference with code that you pointed on:

        public static void InitShaders () {
            var fragmentShader = CompileShader("crate.fs");
            var vertexShader = CompileShader("crate.vs");

            ShaderProgram = GL.createProgram();
            GL.attachShader(ShaderProgram, vertexShader);
            GL.attachShader(ShaderProgram, fragmentShader);
            GL.linkProgram(ShaderProgram);

            bool linkStatus = GL.getProgramParameter(ShaderProgram, GL.LINK_STATUS);
            if (!linkStatus) {
                Builtins.Global["alert"]("Could not link shader");
                return;
            }

            GL.useProgram(ShaderProgram);

            Attributes.VertexPosition = GL.getAttribLocation(ShaderProgram, "aVertexPosition");
            Attributes.VertexNormal = GL.getAttribLocation(ShaderProgram, "aVertexNormal");
            Attributes.TextureCoord = GL.getAttribLocation(ShaderProgram, "aTextureCoord");

            Uniforms.ProjectionMatrix = GL.getUniformLocation(ShaderProgram, "uPMatrix");
            Uniforms.ModelViewMatrix = GL.getUniformLocation(ShaderProgram, "uMVMatrix");
            Uniforms.NormalMatrix = GL.getUniformLocation(ShaderProgram, "uNMatrix");
            Uniforms.Sampler = GL.getUniformLocation(ShaderProgram, "uSampler");
            Uniforms.UseLighting = GL.getUniformLocation(ShaderProgram, "uUseLighting");
            Uniforms.AmbientColor = GL.getUniformLocation(ShaderProgram, "uAmbientColor");
            Uniforms.LightingDirection = GL.getUniformLocation(ShaderProgram, "uLightingDirection");
            Uniforms.DirectionalColor = GL.getUniformLocation(ShaderProgram, "uDirectionalColor");

            GL.enableVertexAttribArray(Attributes.VertexPosition);
            GL.enableVertexAttribArray(Attributes.VertexNormal);
            GL.enableVertexAttribArray(Attributes.TextureCoord);
        }

And code, that will use JsObject API:

    public static void InitShaders()
    {
        var fragmentShader = CompileShader("crate.fs");
        var vertexShader = CompileShader("crate.vs");

        ShaderProgram = GL.Call("createProgram");
        GL.Call("attachShader", ShaderProgram, vertexShader);
        GL.Call("attachShader", ShaderProgram, fragmentShader);
        GL.Call("linkProgram", ShaderProgram);

        bool linkStatus = GL.Call<bool>("getProgramParameter", ShaderProgram, GL.Get("LINK_STATUS"));
        if (!linkStatus)
        {
            Builtins.Global.Call("alert", "Could not link shader");
            return;
        }

        GL.Call("useProgram", ShaderProgram);

        Attributes.VertexPosition = GL.Call("getAttribLocation", ShaderProgram, "aVertexPosition");
        Attributes.VertexNormal = GL.Call("getAttribLocation", ShaderProgram, "aVertexNormal");
        Attributes.TextureCoord = GL.Call("getAttribLocation", ShaderProgram, "aTextureCoord");

        Uniforms.ProjectionMatrix = GL.Call("getUniformLocation", ShaderProgram, "uPMatrix");
        Uniforms.ModelViewMatrix = GL.Call("getUniformLocation", ShaderProgram, "uMVMatrix");
        Uniforms.NormalMatrix = GL.Call("getUniformLocation", ShaderProgram, "uNMatrix");
        Uniforms.Sampler = GL.Call("getUniformLocation", ShaderProgram, "uSampler");
        Uniforms.UseLighting = GL.Call("getUniformLocation", ShaderProgram, "uUseLighting");
        Uniforms.AmbientColor = GL.Call("getUniformLocation", ShaderProgram, "uAmbientColor");
        Uniforms.LightingDirection = GL.Call("getUniformLocation", ShaderProgram, "uLightingDirection");
        Uniforms.DirectionalColor = GL.Call("getUniformLocation", ShaderProgram, "uDirectionalColor");

        GL.Call("enableVertexAttribArray", Attributes.VertexPosition);
        GL.Call("enableVertexAttribArray", Attributes.VertexNormal);
        GL.Call("enableVertexAttribArray", Attributes.TextureCoord);
    }

[kg]

The latter is more verbose and means you can't easily refactor by replacing C# methods with JS methods, etc. It's gross.

The 'dynamic' thing also means that if a user wants static type information they can add it incrementally by replacing a given value with a statically typed one (then all the methods show up, etc). With the GL.Call thing, you'd again have to jump through a ton of hoops - you have to go through and find all the .Calls and replace them with appropriate uses of your static methods, and maybe those static methods have to do Call under the hood, and it's just a mess.

If for some reason we can't make anything else work other than GL.Call etc, well then I guess that's that. But it's pretty terrible :-)

[iskiselev]

I can argue with you about refactoring, that you can introduce const sting with name of Js field, so you still be able to rename it easy - on other hand, you never can't easy refactor dynamic code. Yes, it is a little bit more verbose.
This discussion is very similar to Roslyn discussion of introducing "lite dynamic" through syntax obj.$arg (it should mean obj["arg"]) and final decision to not introduce such feature (see https://roslyn.codeplex.com/discussions/540569, https://roslyn.codeplex.com/discussions/543875 and many more threads about it). There was pretty same arguments on both sides :)

On other hand, if it will be introduced, as you suggested, new attribute, with syntax similar to System.Runtime.CompilerServices.DynamicAttribute, it will work. It is pity, that C# doesn't allow us mark argument/return with DynamicAttribute manually - if it were possible, we was able just mark JsObject with this attribute to use it as dynamic :)

[iskiselev]

And with any decision about dynamic support, @kg, what do you think about introducing JsObject? I've thought about it already when I've written some JS API wrappers :)
I have updated proposed JsObject API a little bit.

[kg]

I'd be fine with introducing that.

[kg]

In fact, file a bug about it. Ideally we can introduce a robust JsObject api, and then the 'dynamic' support for JS objects can be a transform that generates JsObject interactions for you. That way there aren't any behavioral differences between the two approaches, it's just a code simplicity thing.

[ssippe]

So many people have to manually build a bunch of statically-typed bindings for libraries like jQuery to use with other compilers, and that's a huge pile of error-prone busywork.

I wonder if it would be possible to leverage the definitely typed typescript type definitions to elimitate the busy work? e.g. https://github.com/borisyankov/DefinitelyTyped/blob/master/jquery/jquery.d.ts It would be a simple matter of creating a typescript to IL compiler... :-)

[kg]

It's a shame F# type providers aren't available in C#, because you could totally do that with those...

[iskiselev]

Really I also thought about creating some utility that will generate statically-typed wrappers for JS-objects from ".d.ts". Also it would be great if we will be able generate ".d.ts" from translated .Net source, as it will greatly improve integration scenarios.

[kg]

The new emitter infrastructure ilwasm uses could probably be adapted to generate .d.ts files, worth thinking about. I don't know TypeScript very well.

[callanh]

Is there any update on this discussion or any new advice? Otherwise I'm about to start building my own statically-typed bindings for jQuery!

[iskiselev]

@Bablakeluke planned to start implementation of #532. Right now there is no more news/advices. Looks like nobody plan to work on static wrapper based on .d.ts right now.