Quasiquoting based metaprogramming APIs #1989
Comments
mraleph
added
the
static-metaprogramming
|
I think that, overall, this is the best approach. The syntax details may be revisited, but, IMO, quasiquoting is a much more natural and structured way of metaprogramming. In special, I think expression-level macros makes more sense with quasiquoting, although supporting them is not in the current plans. |
|
Could this be covered by string interpolation? Such as: return '''
class Foo {
${someType} myMethod() {}
}
''';This is how various Dart code generators work. This proposal has more advanced features, like for/ifs inside code blocks. But maybe those could be added to string interpolation? Like: '''
class Foo {
${if (condition) 'void doSomething() {}'}
}
''' |
Although you may achieve the same, the approach is different, and string-based generation is way worse in terms of tooling, as we are dealing with, well, strings. |
|
QQ is an interpolation of sorts - it is just interpolation of structured program representation rather than strings. Having QQ does not change expressivity of metaprogramming system, but rather makes it more structured and easier to analyse. Arguably it makes it also easier to read (imagine code highlighting working, imagine jump to definition working etc). Strings are ultimately just sequences of characters, they are incomplete, untyped, unstructured. QQ contents are actual structured pieces of Dart code. |
|
What would the execution model be in Dart for a feature like this? It seems like maybe an interpreter would make the most sense, or did you have something else in mind? I am also curious about how scoping etc would work, do you get a totally new global (compile time) scope for each library? For each outer |
|
I'm skeptical about tooling. AngularDart uses string templates for html, yet it has autocompletion, errors and co. We definitely can have syntax highlighting too. There are a few examples in the JS world where they use string interpolation for things like css or other, yet the strings are highlighted as if it was a ".css" file. |
Would it be correct to say that what you're proposing is a literal syntax for an AST structure, where that syntax
|
|
#1988 is also relevant and could provide a nice way for tooling to treat "strings" as dart code. |
This ends up being a lot more complicated than it might seem on the surface, and its a major sticking point. For the current macro proposal we have ended up defining an entire additional language feature (augmentation libraries) just to solve this problem. We have multiple tools that have to agree when talking about source locations (analyzer, ides, and the various backends and runtimes), so source locations in generated code have to be well defined, and we need to be able to show the user something that actually makes sense to them in a standalone context. This is for instance why we aren't discussing expression or statement level macros seriously at this time. I think this proposal has pretty much the same debugging/tooling issues that those types of macros would. |
I think the execution model is the same as for source based macro-generation: given the source files compiler can extract code that needs to run in the compile time and load that code dynamically (either into the same or a different isolate).
I think all outer There is another related question about scoping (which I kinda skimmed about): how do we define the scope in which Member toJson(Class currentClass) {
// What is the scope in which the below code exists?
return +{
Map<String, dynamic> toJson() => {
-{[
for (var field in currentClass.fields)
+{ -{field.name} : -{ field.ref } }
]}
}
};
}One option is to produce parsed but unresolved ASTs from
That's true to an extent. Highlighting is indeed a relatively simple case. Completions are somewhat harder - note that AngularDart uses constant templates not dynamically generated ones. To do proper analysis on a code template built through string interpolation analyzer would have to understand how to unfold it into the source code - that's much easier to do if you can provide some structural guarantees about how this code is constructed which is what QQ is about.
I think that a possible way of looking at it. The only major difference from string interpolation based API is that the AST structure is constructed in a structured way, rather than as a raw string. Strictly speaking QQ can be to an extent backed by string interpolation ( Member toJson(Class currentClass) {
return Member.fromSource("""
Map<String, dynamic> toJson() => {
${[
for (var field in currentClass.fields)
"${field.name} : ${field.ref}"
].join(',\n')}
}
""");
}Circling back to @rrousselGit point: I think with QQ and string based meta-programming (SB for short) can be viewed as roughly equivalent approaches. However to provide a certain developer experience on top of SB you would need to build understanding of code-inside-strings into the analysis tools - but if you are going to do that, you are just reimplementing a worse version of QQ: hidden inside strings rather than being explicit.
I am not sure it's actually complicated or related to augmentation libraries. Augmentation libraries solve a different problem: they are about generating Dart source into a separate file and having it being spliced into another existing Dart file. What if we don't actually need that property? If splicing is controlled on the AST level, the augmentation libraries are not really needed as a feature. So how would we do debugging? Well, there are several different layers at which you want to debug.
I am not entirely sure what you concern here is. Could you elaborate? Backends and runtimes don't actually care about source locations - they get those from CFE, and CFE can point these to the synthetic source it itself synthesises (so no concern about source location being in sync here). There is also no need to agree between analyzer and CFE because each can synthesise their own source if needed (e.g. to display in error messages or to allow user to open one in the IDE to understand the context of an error message). |
|
I'm not particularly fond of the tokens USD for opening/closing code blocks. If we ignore tooling, Strings are clearer on that aspect.
String templates could be constant too, as we don't have to rely on string interpolation. For example we could do: @Template(r'''
class $name {
void foo() {}
}
''')
class MyTemplate {}voluntarily using MyTemplate(
// parameter generated based on the string content
name: 'HelloWorld',
);This should allow proper analysis |
I am just using We can even use string inspired QQ syntax: Member toJson(Class currentClass) {
return code """
Map<String, dynamic> toJson() => {
${[
for (var field in currentClass.fields)
code "${field.name} : ${field.ref}"
]}
}
""";
}
With the difference that |
|
Similarly, are we sure this would allow us to cover all the use-cases? A common issue with the string interpolation approach is defining dynamic function prototypes/invocations, where the parameters depend on the input code. It rapidly becomes tricky to manage, especially when considering named vs positional and commas. |
There is no reason to worry about separators because you are building it structurally, rather than textually: in other words if you want to build a call you will build a list of expressions which will serve as call arguments, rather than building a string with commas separating arguments. That's by the way another reason why QQ is simpler than building strings. I am not sure I understand the concern about dynamic content: the example with |
It isn't obvious to me how this would work. Say I have the following as input: @annotation
void fn<T>(int positional, {T? named}) {...}
@annotation
void fn2() {...}how would you be able to, from those inputs, have code blocks that generate the following? class FnClass<T> extends StatelessWidget {
FnClass(int positional, {T? named, Key? key}): super(key: key);
}
class Fn2Class<T> extends StatelessWidget {
FnClass({Key? key}): super(key: key);
}Similarly I don't understand how your If so, what if I wanted toJson to generate: class Settings {
Color color;
}
class Document {
Settings settings;
}
// generated:
class GeneratedClass {
void doSomething(Document doc) {
print('doc.settings.color is equal to ${doc.settings.color}');
}
}In this case, how would we get a |
Something along the lines of: Declaration fnClass(FunctionDeclaration fn) {
final name = makeClassName(fn);
return +{
class -{name}<T> extends StatelessWidget {
-{name}(-{fn.parameters.positional}, {Key? key, -{fn.parameters.named}}) : super(key: key) {
}
}
};
}The rule I'd make here is that if you grammatically in the place where you expect a list of something (e.g. a list of parameters) and you have a unquote/splice operation (
Yep: There is actually an alternative to that is we could make
If this is some fixed names you can just write: Declaration makeDoSomething() => +{
void doSomething(Document doc) {
print('doc.settings.color is equal to ${doc.settings.color}');
}
};If you want to use some strings coming from somewhere then we need to add an operation which turns that into a name, rather than a string: Declaration makeDoSomething(String settingsFieldName) => +{
void doSomething(Document doc) {
print('doc.' -{settingsFieldName} '.color is equal to ${doc. -{settingsFieldName.asRef} .color}');
}
};
// makeDoSomething("settings")
Declaration makeDoSomethingEx(List<String> names) => +{
void doSomething(Document doc) {
print('doc.' -{names.join('.')} ' is equal to ${doc. -{names.asRef}}');
}
};
// makeDoSomethingEx(["settings", "color"])(a |
|
I like QQ better than the other metaprogramming proposals for the following reasons:
Additional language features are no longer needed
The only pitfall I see is the lack of complete (resolved AST) introspection, code highlighting & autocompletion. Personally I like the tagged string based QQ or something like The one question I have is about introspection. Currently it sounds like @mraleph is thinking of an AST based introspection, which had a lot of pushback from the dart team in the previous proposals because of language version and language changes potentially causing breakages in the AST that macros might use. Personally I feel like it is just like any package breaking on a change in version of a dependent package, so I'm not as concerned about it. Also are function declarations / classes really going to change that much? You have basic concepts like named parameters, positional parameters, class members, fields, getters, superclasses, etc. If you remove any of those from the language that would be a large breaking change to everyone, not just the macro authors. I believe some simplified form of the AST is relatively stable and is generally useful as an introspection API. However, I am concerned by lacking types for things that have inferred types, which is another advantage of the current proposal. Most getters, setters, fields, and function parameters have explicit types (at least how I write my code), however for many fields I have default initializers or late initializers, and use the inferred type. This I believe to be the biggest problem of QQ, which isn't a problem for languages that are meant to be dynamically typed or which don't use a lot of type inference. I'm am interested in any form of metaprogramming, (current proposal or QQ) and will likely use it a lot in whatever form it ends up. But it seems like if you want to have something powerful and not just a quirk of the language you need AST based metaprogramming / compile time QQ not just annotation based metaprogramming, which is why I pushed a lot to get Code parameters for macro invocations in the current proposal. One of the things that I have heard from the dart team is that it would be ideal if metaprogramming could let library authors create small language features that are specific to their use cases rather than introducing for example 10 different types of value classes. Well I think that QQ might allow for creating more use-case specific language features. Especially because it can be used in expression / statement context in addition to the annotation context, and is not as limited by a class based API. However, if expression / statement macros are included in the current proposal, I see them as roughly feature equivalent, but view the current proposal as more work because of all of the classes and new language features that are required to enable it. However, it is also more powerful because of the introspection on types, and can have more fine grained invalidation and recompilation. In other words I see the pros and cons of both and am glad that there is continued discussion on whether there is a better way than the current proposal with the right balance of features and simplicity. Thanks for all of your hard work! |
There are use-cases that these solve which are not covered by quasiquoting, such as the ability to add initializers to constructors. |
|
@tatumizer It would looks something like this with QQ as I imagine it: class Foo {
-{(){
var list=<int>[2];
for (int i=3; list.length <100; i+=2) {
if (isProbablePrime(i)) list.add(i);
}
return +{ static const smallPrimes= -{ list }; }
}()}
}
I was explicitly opaque on the introspection part. I would like to decouple metaprogramming into two parts: how program structure is understood and how it is manipulated. My initial somewhat naive approaches to metaprogramming were based on Kernel AST manipulation (which gives you both an understanding and manipulation of the program) but over the years I came to appreciate that Kernel ASTs are hidden from Dart users. I think the following should probably be true:
|
|
@TimWhiting, your comparison of QQ to other solutions didn't mention basic string interpolation, which seems to be the closest analog to QQ. String interpolation hits all the same pros that you list. @mraleph, I think your comparisons to string interpolation are too clunky. The following two examples:
and
can be written instead as: Member toJson(Class currentClass) => [
"Map<String, dynamic> toJson() => {",
for (final field in currentClass.fields)
"${field.name} : ${field.ref},",
"};",
].join(); // and let Dart Format take care of everythingEven using |
It's basically the same thing in my eyes. There is a bit of noise around immediate invoked anonymous closure (
That's fair, it's a bit more readable then other string interpolations. Not sure if more readable then QQ though because it squishes different syntactic layers together and has some punctuation Also note that the more imperative you make string-building code the harder it gets to analyse this statically, e.g. with QQ there is a natural (= well defined) way to split declaration building from body building. In your example both are squished together into basically linear code and splitting body building from outline (declaration building) requires some analysis. |
This was my original thought for metaprogramming, but it really doesn't work out in the experiments and discussions we had. You have to somehow provide enough context in the generated file for the user to understand what they are looking at, while also not confusing them. And provide the IDE with real line numbers etc for debugging, setting break points, etc, and also not have the same lines duplicated across multiple files. That is what augmentation libraries are all about, and why they exist as a part of the proposal. It is a general purpose language feature that users can understand, and is also powerful enough to implement macros. It gives us a structured format for presenting the generated code to the user in a readable way. |
|
I'm really glad you filed this, because it's good to talk about it. I'm familiar with quotation and quasiquotations and it's something I've mulled over as part of the metaprogramming story for Dart. I think you are proposing two roughly separable concepts here:
I think it's useful to separate these because they operate at different levels, and we could conceivably do one, the other, or both. Quotation syntaxI don't know about Jake, but I do think quotation syntax is nice. I put some time into trying to come up with a nice design for Dart. One of the main challenges is that Dart's grammar is so complex and syntactically overloaded. Consider: var ast = -{ {} };Is that an expression node for an empty map/set, a statement node for an empty block, an empty class body, an empty function body, etc.? If we only allow quotations to appear embedded in surrounding Dart code, then we can use that context to disambiguate, but I think that would be too much of a restriction for the kinds of complex metaprogramming that our known use cases require. Instead, we'd probably need some kind of marker to indicate what where in the grammar your quotation is for: var ast1 = expr -{ {} }; // Empty map/set.
var ast2 = stmt -{ {} }; // Empty block.Figuring out the full set of these gets hard. Obviously, we'd want expression, statement, declaration, and type annotation. Probably class member, list element, and map element. Parameter? Parameter list? Function body? The more I thought about it, the more I felt like Dart's grammar is just too rich to directly support some kind of syntax literals for it. We are really far from s-exprs. I proposed tagged strings instead because I think they give us most of the brevity of quotations while being a much simpler language feature. They reuse the string and interpolation syntax users already know, which is nice. They let us decide which set of grammar entrypoints to support at the library level instead of baking into the language. And it's a general purpose language feature that could be used outside of metaprogramming. Tagged strings probably don't give as nice of an IDE experience as real quotation syntax would—syntax highlighting, parsing, etc.—but I have some optimism that that is doable even with tagged strings. A Dart IDE can statically resolve the tag name and if it points to our own metaprogramming library, then a smart IDE could possibly parse and syntax highlight the contents of the string. Strings also give users the luxury of composing code out of fragments that are themselves not complete grammar productions. For example, a pair of arguments is a meaningful thing to want to interpolated into some generated code, but it's not a meaningful syntactic unit in the Dart grammar. I realize semi-stringly-typed programming isn't a totally first-class developer experience, but I think the simplicity of it does have some value. Directly embedded quotationsThe metaprogramming part of your proposal is that if quotation appears where code is expected, the quotation is run at compile time and the result interpolated. Almost as if the entire library is itself a quotation that can contain unquoted expressions. I really like how easy this would make it for users to insert a little one-off metaprogramming in their code. But my understanding is that those kind of uses are a minority of the use cases we have. The stuff users are asking for is mostly reusable metaprogramming. Things like JSON serialization, data classes, auto-disposable, etc. My expectation is that most metaprogramming would be "framework-like" code in that a single proficient author will write a macro that gets consumed by many users. In that model, what I think matters most is that applying metaprogramming feels as easy, simple, and seamless as possible. Given: -{ import 'package:json_serializable/macros.dart' }
class Foo {
final String a;
final String b;
-{toJson(currentClass)}
}This requires users to understand the quotation syntax, and that some kind of special imports are required. Compare that to what we're aiming for with the macro proposal: import 'package:json_serializable/macros.dart'
@toJson
class Foo {
final String a;
final String b;
}This is already perfectly valid, familiar Dart code today. It's the code users already write when they are using a library that does code generation. Macros just provide a more integrated way to implement that. I'm also worried that requiring all metaprogramming code to live inside quasiquotations would make it hard to do metaprogramming in the large. Many of the use cases we have are a lot more complex than just synthesizing a statement or two. Metaprogramming authors need to do fairly complex introspection and build up function bodies using a decent amount of imperative code. Because of that, I want it to be easy for authors to split their metaprogramming into multiple functions, reuse code, etc. That gets harder if everything has to live inside immediately invoked closures. A big part of this is that our use cases for metaprogramming in Dart are quite different from metaprogramming in other languages. In languages like Lisp and Scheme (and maybe Rust?), you're often using metaprogramming to add new small-scale syntactic niceties. Macros are for extending the imperative syntax of the language and making. Dart is already quite expressive at the statement and expression level (for better or worse), so we don't have many use cases for new syntax. There's only so many control flow structures one could want and we have a whole lot of them already. Most of our use cases are really about procedurally generating entire declarations. It's about code reuse and programmability at the declaration level. That requires metaprogramming that has fairly deep introspection ability, access to typed and resolved elements, and the ability to write pretty complex macro code. That's why we're leaning towards an approach where macros are full-featured Dart classes. But, having said that, I wouldn't rule out some kind of quotation syntax. I pitched tagged strings because it's simpler (and can be used for other purposes), but other options are definitely on the table. |
|
As much as I like the idea of Quasiquoting, I've been convinced that it is not the right fit for the metaprogramming side of things because of the introspection / type information that you can get from the current proposal. Also I appreciate that augmentation libraries and tagged strings solve more use cases than just metaprogramming. However, I still don't like the idea of generated files all over the place, but it is something I can deal with (since that is the way we have it currently & we can hide them from the IDEs etc). |
Just to clarify a bit, the augmentation library approach was introduced specifically to solve this exact problem (the problem of how to surface macro generated code to the user). The original approach we proposed was to do exactly the style of splicing you are discussing here, and very strong concerns were raised about this from the teams who would be tasked with implementing this, around how to give a good UX to end users. Augmentation libraries were proposed as a way to provide a unified metaphor for how to surface macro generated code to the user across our tools. |
|
I think I have managed to conflate a lot of topics into a single issue due to the complicated nature of the topic. Lets try to split it out a bit: my main concern is about generating structured data (Dart code) from other structured data (Dart code) through an unstructured medium - raw text. What I am trying to argue is that there are strong benefits in using structured medium instead - QQ syntax. One interesting one that I have brought up before is that you can statically split a single QQ based macro into "outline" only and "body" parts without programmer actually doing it manually. Other things that we discuss are tangential, and are not related to the direct core of the proposal. QQ can be used in macroses that are triggered using annotation syntax and QQ can be used to generate augmentation libraries.
I have considered this problem before and I think we can just rely on types, e.g.
I am not sure where my proposal implied that this has to be done this way. I have written some code inline in a QQ but I have also indicated that you can move it to a separate file if you want. You can split and structure your code in any way you want. QQ is just a way to move between the levels. It has no implications on how you would organise the macros code (instead that you would have to organise it in a structural way). Member toJson(Class currentClass) {
return +{
Map<String, dynamic> toJson() => {
-{[
for (var field in currentClass.fields)
+{ -{field.name} : -{ field.ref } }
]}
}
};
}
// Can be written as
Tree _serializeAField(Field field) =>
+{ -{field.name} : -{ field.ref } };
List<Tree> _serializeFields(Class currentClass) => [
for (var field in currentClass.fields) _serializeAField(field)
];
Member toJson(Class currentClass) {
return +{
Map<String, dynamic> toJson() => {
-{_serializeFields(currentClass)}
}
};
}
I don't think Dart is unique in how we plan to use macros. Generating boilerplate code and declarations is a common use case for compile-time metaprogramming features in languages that have complicated ASTs. |
It might be worthwhile to write a short side-by-side comparison of the two approaches (not necessarily in this issue) using one of those complex use cases. That might make the conversation a bit more concrete. |
|
If there are existing examples using string based code APIs - I would be happy to port some to QQ myself. |
|
(Have to admit I didn't read all the comments yet) I understand that switching to QQ API completely might be hard. But could we at least keep the current API but add some special syntax for So instead of writing builder.declareInType(DeclarationCode.fromParts([
'external ',
// ignore: deprecated_member_use
await builder.resolveIdentifier(Uri.parse('dart:core'), 'int'),
' get hashCode;',
]));we could write builder.declareInType(+{ external int get hashCode; });or, as a more complicated example, instead of the long function body here we could writebuilder.declareInType(
+{ -{ clazz }.gen( -{ superconstructor.parameters } )
-{
switch (superconstructor) {
null => +{},
_ => +{: -{ superconstructor } ( -{ args } )}
}
};
});(which of course also requires calculating |
Actually, they can produce any kind of object :). So this example (modified to assume
Could be something like the following: builder.declareInType(declaration 'external $int get hashCode;');I think that generally accomplishes what you are asking for, while being a generally useful feature for other purposes. Essentially, you can think of it as just a shorthand for passing lists of strings mixed with other values into a function. |
But why this won't just be an error? Splice pastes one piece of AST into a bigger one, here not only you are pasting a statement into RHS of an assignment, but also the code can't be correct for any splice because of missing semi-colon. |
Right, I looked at the proposal. But the point is produce here. What bothers me is they still take strings. So, not only there will be no syntax highlighting in IDEs, but it's also super easy to mess things up. I actually didn't mean |
They can be nicely typed though, so in the case of the Code apis, they would only accept other Code objects for the "thunks". It will be easy to just assume things from See also the reflected imports proposal to make it easier to get something like "normal resolution" for identifiers that a macro knows they will want (and have the ability to import). I am not convinced this is actually a good proposal though because it implies macros need access to the API of all transitive reflected imports to compile them, and those could require platform specific code that we don't want to be available to macros. So some other mechanism is probably better, although it would work OK for basic cases. |
|
Thanks, I've missed the nicely typed part. And I guess if hand written parser fails at compile time, with some effort on the IDE side we could propagate the error back to the string literal. I agree it's mostly achieves what I'd like to achieve. I also agree that it's a more general solution, since users could provide parsers for their own grammar. FTR, Template Haskell's quasi-quoters, mentioned in the issue description, are working exactly like tagged strings proposal (with the only difference that they inherit the host language lexer, so they work with lists of tokens instead of strings, but each quasi-quoter still has to do parsing on its own). OTOH, having to use that for the host language grammar feels kinda like poor man's quasi-quoting. Personally I'd appreciate having true QQ for Dart syntax alongside with tagged strings. |
|
@munificent I think you generally mix usage of Top-level unquotes (splices, Some examples: -{myMacro()} // myMacro must return a top-level declaration/definition or a directive.class C {
-{otherMacro()} // otherMacro must return a member declaration/definition
}var x = -{exprMacro()}; // exprMacro must return an expressionQuotes ( |
|
There's a lot of great discussion here, thanks! As far as I can tell tagged strings #1988 is the main actionable proposal, if there is anything else we might do please reopen / file as more specific issues. |
github-project-automation
bot
moved this from Backlog
to Done
in Static Metaprogramming - design/prototype
|
I have tried to use the current incarnation of the macros and I think that current code generation API leaves a lot to be desired. I don't think they are easy to read and write: final commonTypes = await _resolveTypes(builder);
final implName = '_${clazz.identifier.name}Impl';
builder.declareType(
implName,
DeclarationCode.fromParts([
'class $implName implements ',
NamedTypeAnnotationCode(name: clazz.identifier),
' {\n'
' final ',
commonTypes.arena,
' _arena;\n',
' final ',
commonTypes.pointerUint8,
' _data;\n',
' $implName._(this._arena, this._data);\n}\n'
]));And the best thing you can do to make this readable within the current paradigm is to split it into smaller chunks. Text centrism also gives raise to lots of hard to figure corners in the APIs (e.g. how do I resolve a type? how do I substitute a type reference into the code, etc). Needing to remember The quasi-quoting gives answer to all these and more. Dart syntax becomes an API for generating Dart syntax. final implName = '_${clazz.identifier.name}Impl';
final type = declare {
import 'dart:typed_data';
import 'package:simdjson/src/rt.dart' as rt;
class ${implName} implements ${clazz} {
rt.Arena _arena;
Pointer<Uint8> _data;
${implName}._(this._arena, this._data);
}
};I am using I don't want to outright reopen this issue - but I strongly feel we should reconsider the current state of things. |
|
tagged strings would significantly improve the ergonomics of creating code objects. |
|
Would tagged strings allow for internal string highlighting and static analysis? |
No they would not, but macros are not generally written as a single giant declaration in my experience. They are built from pieces of shared generated code with lots of imperative validation code mixed in. So, you aren't going to get much syntax highlighting from quasi quoting either I don't think in practice. |
|
@mraleph I think the |
|
IMO we should have some form of string template. For comparison, the following: final commonTypes = await _resolveTypes(builder);
final implName = '_${clazz.identifier.name}Impl';
builder.declareType(
implName,
DeclarationCode.fromParts([
'class $implName implements ',
NamedTypeAnnotationCode(name: clazz.identifier),
' {\n'
' final ',
commonTypes.arena,
' _arena;\n',
' final ',
commonTypes.pointerUint8,
' _data;\n',
' $implName._(this._arena, this._data);\n}\n'
]));Becomes: final implName = '_${clazz.identifier.name}Impl';
builder.declareTypeString(
implName,
args: {'classId': clazz.identifier},
'''
class $implName implements {{classId}} {
$implName._(this._arena, this._data);
final {{package:arena/arena.dart#Arena}} _arena;
final {{package:typed_data/typed_data.dart#U8intBuffer}} _data;
}'''); I'm sure we could have a package for this. |
|
Tagged strings can support anything tools can figure out to support Declaration ids(int count) =>
decl"var ids = [for (var i = 0; i < $count; i++) "id\$i"];";could not be recognized and highlighted. (But maybe we want it to be highlighted slightly differently from non-template code, otherwise it could be harder to see what's going on.) |
|
I would like to show an example of the T4 syntax from C#. <#@ template language="C#" #> // Useless in Dart, .NET has more languages
<#@ output extension=".cs" #> // T4 generates files on disk, like build_runner, which is a bad DX.
<#
// Define a list of properties for the class
var properties = new List<Tuple<string, string>> {
Tuple.Create("int", "Id"),
Tuple.Create("string", "Name"),
Tuple.Create("DateTime", "DateOfBirth")
};
#>
// Auto-generated class
public class GeneratedClass
{
<# foreach (var prop in properties) { #>
public <#= prop.Item1 #> <#= prop.Item2 #> { get; set; }
<# } #>
}Output: It's almost pure C# code. It's more readable than the first syntax with + and - around and provides the same benefits. I feel like <#
// Define a list of properties for the class
var properties = [
('int', 'id'),
('String', 'name'),
('DateTime', 'dateOfBirth')
];
#>
// Auto-generated class
class GeneratedClass {
<# for (var prop in properties) { #>
<#= prop.$1 #> <#= prop.$2 #>;
<# } #>
} |
|
Another possibility is to introduce a unary operator final out = StringBuffer();
> 'class $implName implements ${NamedTypeAnnotationCode(name: clazz.identifier)};';
> '{\n';
> ' final ${commonTypes.arena} _arena;\n';
> ' final ${commonTypes.pointerUint8} _data;\n';
> '}\n';
builder.declareType(implName, out.toString()); |
|
Using plain strings does not get your syntax checking. It's not putting into a Using a special operator with a hard-coded receiver name requires a language feature, and it's not wildly more readable than out + ' ....';
out + '.....';with Or even: extension on String {
void operator~() {
((Zone.current[#_out] ?? (throw StateError("No current output"))) as StringSink).write(this);
}
}
R withOutput<R>(StringSink output, R Function() action) =>
runZoned<R>(action, zoneVariables: {#_out: output});which you use as: var buffer = StringBuffer();
withOutput(buffer, () {
something;
~"text to output ${something} or else\n";
~"more text to output;
});But none of those are as readable as |
|
I think readability is more important than guaranteeing the validity of the generated code here. Badly generated code will lead to a well contained compile error and a bug report on the macro. It can get fixed quickly. On a different note, one thing I was considering was writing a macro that's applied either on a String literal, or on a random Dart class/function/... For example, we could have: @Template('myTemplate')
class $Name extends MyBaseClass<$BuildT> {
$BuildT build() => throw UnimplementedError();
}Which generates code that can be used as: builder.writeTemplate(
myTemplate(
args: (
name: 'Example',
buildT: Uri.parse('dart:core#int'),
),
/* some optional args, to maybe enable adding mixins or methods in here */,
),
);
// Will generate:
class Example extends MyBaseClass<int> {
int build() => throw UnimplementedError();
}Obviously the exact API needs refining. But I think that could be an alternative way to get readable generation and sane ouput. |
|
For readability, every piece of noise counts. builder.declareType(implName, code ((out) {
~ 'class $implName implements ${NamedTypeAnnotationCode(name: clazz.identifier)} {';
~ ' final ${commonTypes.arena} _arena;';
~ ' final ${commonTypes.pointerUint8} _data;';
~ '}';
});@rrousselGit : Creating a template for a single use is too much hassle with no clear benefit. For every program with a template, you can write an equivalent cleaner program using the above notation (IMO).
I agree with that wholeheartedly :-). Chasing too many rabbits is rarely an advisable strategy :-) |
|
@tatumizer I already use something similar: #1989 (comment) What's being discussed is a way to be more safe with regards to what's generated. Like preventing missing/extra Quasiquoting, or even Templates would be more type-safe. It can do things like only allowing to write a constructor in a class/enum. |
This is impossible in a general case. The fragments of the program can be generated conditionally, or in a loop - they won't exist as a single piece until joined together. Anyway, the resulting overall output will be shown in UI, with a normal highlighting and errors. Frankly, I don't see a problem worth solving here. |
Right now a lot of metaprogramming discussion is centered on generating the source and consequently string based metaprogramming API. I think there might be an alternative approach which is much more structured and instead based on quasiquoting, similar to existing metaprogramming systems (LISP dialects, Template Haskell, Scala, Metalua, Converge).
In a nutshell quasiquoting (QQ for short) mechanisms provide a way to move between layers (e.g. compilation and execution can be seen as two layers) in a structured way.
Example
Here is how it could look like:
I am using here
-{...}and+{...}syntax for quasiquotes which originates from Metalua (one of the systems I had a chance to extensively work with in another life), but I am not fixed on any particular syntax as long at it allows to express at the very least the following concepts:+{code}shifts up a metalevel (i.e. from compilation to execution), essentially it evaluates to an abstract representation of the given code fragment.-{code}shifts down in the metalevel (i.e. from execution to compilation), essentially given an abstract representation of the code fragment it splices this fragment into another one.It might be that we would need to introduce few different QQ-variants for different context (and potentially to simplify parsing rules - which otherwise might need to use types for figuring out how to parse things, which might not be desirable).
Note that in the example above I combine both compile-time and execution-time layers in a single file. This might not be desirable for performance reasons. In which case the following could also work:
Lets look at more examples:
Pros of quasiquotation
Member x = +{1 * 2}compiler can tell you that1 * 2is not a valid AST for a member and instead is anExpression).toJsonfunction from the very first example compiler has a natural way of splitting it into outlining phase (which generates just the member signatures) and body generation phase (which generates member bodies).Cons of quasiquotation
QQ is built around abstract representation of the source, rather than the source itself, so it might be harder to debug than source based solution. On the other hand there is a natural mapping from QQ constructed ASTs to source code on the disk so for debugging purposes it should be simply possible to dump that and leave it to user's inspection.
Prior Art
/cc @leafpetersen @lrhn @jakemac53 @munificent
The text was updated successfully, but these errors were encountered: