Allow type providers to generate types from other types [ RFC FS-1023 ]

[baronfel]

Submitted by Tracy on 3/24/2014 12:00:00 AM
116 votes on UserVoice prior to migration

• RFC Added

There are occasions where it would be extremely useful to generate types from other types.
As an example, F# interop with NHibernate is very clumsy simply because it's difficult to express types of the sort:

// C# record class
public class MyRecord
{
    public virtual int Id { get; set; }
    public virtual string Description { get; set; }
    // etc...
}

It would be very compelling to be able to represent these as F# record types, but the CIL code generated for F# records is incompatible with NHibernate.
Perhaps it could be possible, using a type provider, to generate the POCO class above from an F# record type of the sort:

type MyRecord = { Id : int, Description : string }

The type could be generated as shown below:

type MyPocoRecord = PocoTypeProvider<MyRecord>()

I understand the difficulty of doing this at compile type.Tomas P actually explained why in a forum post (that I can't seem to find.) However, this sort of problem is the reason by the CLIMutable attribute was created, which as far as I can tell, was hard-coded directly into the F# compiler. I can see these interop dilemmas becoming more common as F# adoption increases, especially in the enterprise where tools like NHibernate are in widespread use. There ought to be a way to address them without creating one-off CLIMutable-esque attributes per se.
The feature itself would open the door to incredibly powerful metaprogramming opportunities.

Response

** by fslang-admin on 6/24/2016 12:00:00 AM **

Marking this as “approved in principle” per comment below.
However it will be a difficult feature to land in practice and will be subject to very many caveats and likely limitations. There’s no certainty that this will make it into F#.
We will open an RFC for it eventually (it won’t be fast :) )
https://github.com/fsharp/FSharpLangDesign/tree/master/RFCs
Don Syme
F# Language Evolution

Original UserVoice Submission
Archived Uservoice Comments

[kurtschelfthout]

RFC was added.

[robkuz]

Will this allow for generic types coming out of the provider

type Bar a = MyTP<Foo>

?

[dsyme]

@robkuz No, that's tracked by a different suggestion

[robkuz]

@dsyme
by which suggestion is that covered? I couldn't find anything.

Btw. Why is this? In general it is possible to dynamically create parametrized generic types? One could even do that from todays TPs using https://msdn.microsoft.com/de-de/library/system.reflection.emit.typebuilder.definegenericparameters(v=vs.110).aspx

[dsyme]

RFC is here: #212

[voronoipotato]

#450 nearly certainly depends on this according to dsyme via uservoice should it be accepted.

[robkuz]

T-TPs should support the creation of DUs.

I have the following to 2 use cases when working in Fable to create strongly typed UIs.

type SourceDU =
| NoParamCase
| SingleParamCase of SomeType
| TwoParamCase of SomeType * AndAnotherType

Now I want to create a derivate for this DU that is consisting of only NoParamCases so that

type TargetDU =
| NoParamCase
| SingleParamCase
| TwoParamCase
with
     static member OfSourceDu (x: SourceDU) =
          match x with
          | NoParamCase ->NoParamCase
          | SingleParamCase _ -> SingleParamCase
          | TwoParamCase _ -> TwoParamCase
     static member Enumerate =
          [ NoParamCase; SingleParamCase; TwoParamCase]
     override this.ToString () =
          match this with
          | NoParamCase ->"NoParamCase"
          | SingleParamCase -> "SingleParamCase"
          | TwoParamCase -> "TwoParamCase"

Usually this is needed when constructing a SourceDU case in the UI where one selects the target DU case from a combobox and the shows different UI elements depending on the selected item in the combobox.
Atm handling this kind of situation requires lots of boilerplate and close attention for compiler warnings if something has been added to the SourceDU

The second example concerns the strongly typed updating of records in an elmish architecture.

given then following records

type Address = {street: string; city: string}
type User = {active: bool; age: int; address: Address}

I'd like to create the following DUs

type AddressUpdateMsg = 
| Street of string
| City of string
with
     member this.Update (x: Address) =
         match this with
         | Street v -> {x with street = v}
         | City v -> {x with city = v}

type UserUpdateMsg = 
| Active of bool
| Age of int
| Address of AddressUpdateMsg
with
     member this.Update (x: User) =
         match this with
         | Active v -> {x with active = v}
         | Age v -> {x with age = v}
         | Address msg -> {x with address = msg.Update(x.address)}

Again all of this can be easily derived from the initial 2 type definitions without any involvement from the developer.

[7sharp9]

@robkuz This one would have to be completed first to allow this:
#154

[robkuz]

@7sharp9 yes ths should be supported as well ;-)

[7sharp9]

@robkuz You could also generate both of those with falanx too, we currently generate unions and records from a schemata or whatever you wish.

[robkuz]

yeah, but you generate those from an external schema ... however I want to generate them from an internal type ...

[7sharp9]

The schemata come from whatever you want it too, can come from an ast fragment or lump of text or anything else 🍌

[robkuz]

@7sharp9 sure enough - however there is no std. way to parse an exisiting code base in F# and also to generate code into it. Something that Ocaml and also Kotlin have figured out.
There is also no default hooks to look for when parsing.
So you are right in theory but in practice ...

[et1975]

@robkuz I think the idea is not parse it from the code sources, but using a bit of F# reflection, which would be relatively easy.

[7sharp9]

@et1975 Gets a bit tricky when you need to use reflection and the types do not really exist like providedtype derivatives. I had to jump through a few hoops to get ProvidedRecords and ProvidedUnions working in Falanx.

[voronoipotato]

@7sharp9 is a good resource for tackling this due to his experience with Falanx. I suspect there's a reason that we use the AST for example in Visual Studio extensions. It's hard to beat, even though there can be a bit of a learning curve with parsing I think that is something that good editor features can mitigate so that even beginners could tackle it. I'm okay with type providers using AST as we require the same thing for VS extensions to the best of my knowledge. My coworker with no compiler experience was able to figure it out for VB and give us an automatic code-fix which has been incredibly valuable.

[7sharp9]

The biggest usability issue in that area is its not that east to parse a fragment of ast, you have to run it through part of the checker to get a parsetree. It would be really good if these could be literally quoted but my suggestion for that was marked as probably not See #682

[voronoipotato]

I think more use cases like this help give a case for the value of AST code quotations.

[Luiz-Monad]

....
Again all of this can be easily derived from the initial 2 type definitions without any involvement from the developer.

This would allow for something like.

data  Maybe a  =  Nothing | Just a
  deriving (Eq, Ord)

If TypeProviders could do that, this would increase the ranking of the language (pun intended).

But the problem would be chiken-and-egg when compiling, unless there's some way of specifying double-pass in the analysis (or in the build step).
Also, noting that the F# project already require files to be defined in order to be compiled, so I don't think that would be that much of a problem.
It would naturally lend itself to something like:

///def.fs
type MyRecord = { Id : int, Description : string }

///usage.fs (after def.fs in the project)
type MyPocoRecord = PocoTypeProvider<MyRecord>()

That would be bad for the build system though.
But I really liked this idea, it would open a really solid way of meta-programming.

[7sharp9]

Incidentally you can generate F# types from F# types with myriad

[Luiz-Monad]

That's what I ended up doing, It was simpler than writing the Type Provider.

[albertwoo]

I think this is very useful for Bolero and Fabulous project, so we can generate types for third party libraries very easily. Hope this can be done soon.
fsbolero/Bolero#142

[kerams]

How is the compiler on .NET Framework (in VS) supposed to load the assembly of the input type if the TP consumer targets .NET Core? I guess .NET Core could become a requirement for every part of the pipeline, but that would also mean VS would have to move to out-of-process hosting of the compiler first.

Or perhaps it's possible to create fake Type instances based on metadata contained in TyconRef?

[vzarytovskii]

How is the compiler on .NET Framework (in VS) supposed to load the assembly of the input type if the TP consumer targets .NET Core? I guess .NET Core could become a requirement for every part of the pipeline, but that would also mean VS would have to move to out-of-process hosting of the compiler first.

Or perhaps it's possible to create fake Type instances based on metadata contained in TyconRef?

We already support both full framework as well as coreclr compilers in VS.

Another problem may be FCS we host in VS, which will be full framework for near future.

[T-Gro]

Considering the .NetFramework (IDE tooling VS) vs netcoreapp (user code) communication for F#-defined types.
Couldn't a TypedTreePickle-derived format be used for the exchange? The inspection would then need to stop as it encounters an IL type - reference to it could be still kept around, but the TP would not be able to inspect the inner contents of an IL type (let's say a BCL type only coming from NET9, which the VS tooling cannot load) . But it would still give the power to manipulate and augment F#-created types.