Support yaml/json #16
Comments
|
I don't believe yaml is in fact markup, but I'm all for json... Much clearer. |
|
json +1 |
|
json +1 here |
|
json +1² |
|
JSON would be totally in spirit of the "new" .NET, +1 |
|
I'd be curious how a json formatted MSBuild project file would play with AspNet vNext's project.json. |
|
json support will be great of course after getting msbuild xplat |
|
One important bit about json config is the (lack of) comments,which are useful in a build configuration. How would json support that? |
|
@hmemcpy the fact that official JSON standard doesn't support comments doesn't mean a thing. JS comments can be easily handled by a JSON parser.It doesn't need to be "Crockford's valid JSON", you know. |
|
I think it would be helpful if someone converted an existing MSBuild solution (e.g. this repo) to json so we can see how it looks like :) |
|
Without being for or against this suggestion, the main thing we would need to keep in mind is alignment with Visual Studio and the project files. It's something we will definitely bring up for Visual Studio vNext, but I'm not sure that we would take a change right now for the current MSBuild incarnation as it would likely be fairly substantial. |
|
👎 Unless someone plans to write a JSON 'schema validator' this will not happen. XML has all that built in. A better choice would be some JSON to MsBuild XML preprocessor task. Well anyone can do that, it does not have to be built into MsBuild. |
|
@leppie |
|
YAML is a superset of JSON, so I'd rather see support for YAML (which would thus automatically include JSON support unless YAML-specific features like multiple documents in a single file are required). YAML is also a lot easier to hand-modify in case that matters for people (comes up for me very occasionally where I need to manually edit a .csproj because I accidentally did something derpy in VS that I couldn't fully revert). |
|
@darrelmiller, I know what it means, but it's not really 'markup'. Just my opinion, but I'm not a fan. Invisible characters have no place as deliminators. Json support on the other hand aligns nicely with the vnext direction. Also, extra data inside json wouldn't affect its purpose, so comments can easily just be extra data. When parsed, comments show up as properties. |
|
@ChaseFlorell I deleted my comment because I was wrong about what YAML stands for 😮 |
|
I hate to be the lone ranger here. :) (actually, I think @ChaseFlorell can back me up here if he is who I think he is. 😄). But I would LOVE to see MSBuild support Xaml. Xaml already has built-in designer and tooling support in VisualStudio, and has rich support for intellisense through add-ons such as ReSharper. Not to mention, it is used everywhere in MSFT ecosystem these days (except ASP.NET, but I personally hope to see that change that as well. 😄). Xamarin is also blowing up with a bunch of Xaml adoption as well. It's pretty ubiquitous and should definitely be considered here. Xaml is thought of as presentation markup, but it is an object definition language that can be used anywhere. Again, it comes with built-in rich tooling support for free. As a quick example, here are some POCOs that I've designed: And here is a a corresponding simple Xaml definition: When I select the root element with my mouse, this is what I see in the Properties Pane: All for free, right now in Visual Studio. If you have ever worked with Blend, then you know those controls are Blend designer controls and comes with all the designer/tooling support with them (for instance, that enum is a dropdown with all of its supported values). We can even create our own custom editors for properties if we so desired (and once the community grows, so will this desire, I'm sure :)). Finally, by using Xaml, you get to take advantage of the one unique component that nothing else in the entire world uses: MARKUP EXTENSIONS! This is what makes Xaml so incredibly awesome and fun to work with. I know some people find XML/Xaml as verbose, but I (and others) find it expressive and powerful. I would also like to say that Xaml is NOT Xml. It is its own animal altogether and should be considered separately in its own right, IMO. I would LOVEEEE to have this functionality with MSBuild!!! |
|
Some additional considerations as well: XML was intended and designed to be language-agnostic. I know that JSON is the new, shiny kid on on the block, but it violates this principle this just by its very name. Also, the magic schema it is using is (in my view) suspect, as it introduces dynamic (unstructured/unknown/loose) properties that may or may not be consumed/understood/known by its consumers, possibly causing inadvertent side effects. Finally, and this is nit-picky, but it also introduces unnecessary overhead to the file (in that it introduces data into the file that is intended to only be known by designers, that now potentially goes over the wire), which flies in the face of its "lightweight/optimized" value proposition. Furthermore, when using the current JSON schema mechanism, it references another artifact that lives on the system somewhere. So now, from a tooling (and management) perspective, we have to account for 2 artifacts instead of one. This is exactly how the .NET configuration mechanism worked and it was a burden to work with (even more so as it required you to create configuration mapping items altogether to map between the POCO and configuration elements, but I digress). Using the class definition as the schema (as Xaml does) is the way to go, as you only end up dealing with one artifact and all tooling is derived from there (the class definition). You are also ensured of proper, consistent, and congruent property casing which was also another annoying side effect of .NET xml configuration (and JSON Schemas) [e.g. "PropertyName" is defined in the class file while "propertyName" is defined in the Xml/JSON file]. Once JSON uses class definitions as schemas, then I feel it can start to be considered as a legitimate object definition "markup" (please kindly recall that JSON is actually code) language. But then, at that point, it's being exactly like Xaml... so why not go with the option that has the more history/maturity and (most importantly) tooling support (this is what makes us efficient, after all!). ... and then there's that whole language-agnostic thing... oh, and markup extensions. :) I'm not trying to start a JSON vs Xaml fight (ok, maybe a little 😇), but I am trying to make sure all points of view are considered here before we all go chasing the new shiny toy. Some of us have been around a while here and there are reasons why we do the things we do... so it's good to discuss these things and get them out and have all points considered. :) |
|
I actually like the idea of XAML over JSON. If I had to write MSBuild configurations in JSON, I would probably want to jump out a window. |
|
@Michael-DST
From json.org: "JSON is a text format that is completely language independent." (In fact they are both their own languages, hence the L in XML; but they are markup languages rather than general purpose languages.)
Not sure what "magic" you are talking about. Just like XML, JSON can be schemaless, or it can be tied to a schema.
Not sure what you are talking about here; what "designers"?
In both cases, there is a schema definition that lives separately from the markup; in XML/JSON its a schema file, in XAML its a class, but there are no fewer artifacts to deal with by using XAML. (Not to mention, doesn't using a class for the schema definition destroy the "language agnostic" characteristic you were advocating for?)
No, it's not. It may have started out that way, but [json.org](language definition for JSON) clearly specifies that only objects and scalar values are valid; arbitrary expressions are not allowed.
Xaml may have more history, but JSON is and will always be more widely supported in the software industry (and the same is true for XML). I don't know that I want a new developer to have to learn XAML AND MSBuild just to understand how our build works (see my following comment on build DSLs).
Tooling support - especially visual tooling, which seems to be what you are talking about - is definitely NOT the most important thing in a build tool. I value visual designers when they are appropriate, but a point-and-click designer for a build tool would just slow me down compared to a fast code editor with Intellisense. The thing is, I am not necessarily opposed to XAML over XML for MSBuild. I used WPF heavily for several years, and really appreciated the power and expressiveness of XAML. But it is verbose, and it is complex, and there is a steep learning curve compared to most markup languages. XAML would have to provide significant benefits for the build system to be worth considering over XML, especially given that MSBuild needs to be approachable by any developer on any .NET project. In my opinion, you haven't provided any significant evidence of such benefits. |
|
I think it is worth considering that markup is not the language of choice for modern build systems. MSBuild was created at a time when Ant, NAnt, and Maven were some of the most popular build systems. But today, Gradle, Rake, and Grunt are becoming the tools of choice, and for good reason. Build systems based on code rather than markup use DSLs to make doing the simple things easy, while making the hard things possible, without jumping completely outside of the build system to do them. My own hard experience with both MSBuild and Maven has shown me the pain that results from pretending that the build isn't a software system of its own. We should treat it as such by using the best tools we have available for building software. My opinion is that the best option for a new language for MSBuild would be the new C# scripting abilities being built into Roslyn. |
|
Cool. Thank you for the discussion @eatdrinksleepcode. :) I knew I was gonna rile someone up after I posted all that... I almost deleted it haha. To start with, when I say language-agnostic, I mean it is agnostic from any other language from itself. JSON's (original) intent is to be consumed by a JavaScript or JavaScript-aware client for serialization/conversion into a JavaScript application. I realize it can be used in any application, but I am just looking at it from the perspective from other non-JS languages. To me, it's about the equivalent as seeing a VBON (VB) or CSON (C#). Sure, those would make sense in their two respective paradigms, but outside, of it... well, that's where we run into this area. (To be sure, I am not saying those exist... but just saying that's what it is like.)
More succinctly, the consumers of the file format that provide design-support. In this case, MSBuild (or Visual Studio). When I say "magic" I am referring to the "$schema" attribute. If I understand correctly, this points to another artifact altogether to that is the schema to describe the data, correct? That means that 1) someone somewhere has to create that artifact at some point and 2) know where it is and reference it in their file. The schema in Xaml is inferred from the class itself, so there's no other external artifact than the current file that you need to worry about. That makes it easier to create/manage/maintain/etc.
Xaml is also still language agnostic in this case, as the class it is pointing to could have been compiled as C#, or VB, or even F#. That is what is (one of the many things) awesome about it, IMO. :)
Well, I am not so sure about this. There is so much to be done in this area, innovation-wise. But at this point I am mostly talking about intellisense. And the intellisense with Xaml (especially with ReSharper) is so much better than what is offered with Xml and JSON (at the moment, I'm sure it will get better). The fact that so much has been done around its format in Visual Studio for presentation tooling (in WPF, etc) makes it all the better (again, I think there is so much innovation left to be done in this area to get to what I am talking about).
I would like to make sure here that "learning Xaml" isn't being distorted by your perception of how complex WPF is. They are totally separate. Xaml is just XML, and I think as far as its object definition qualities go, developers can jump right in, as they can leverage the ubiquitousness (that's a word haha) of XML. With all of this said, I am 100% with you about Roslyn and its new tooling. What I personally would like to see done is a new "Data AST (Abstract Syntax Tree) Format" that is a new file format (call it .dast for the sake of this discussion). When this file is opened in Visual Studio, it displays it to the developer in the format that the developer prefers it (JSON, XML, Xaml, Yaml, etc etc). That way, the data gets saved (in a particular format, which may or may not be proprietary), and then whenever it is opened, it is presented in a preferred format, and is designed so accordingly. Not sure how viable that is short-term and what that means for MSBuild, however. :) The thing you are fighting with me here on is that I (and you, it sounds) know what the Xaml development/designer experience is like, and it's quite nice (especially with ReSharper). JSON development/designer support is going to be a while before it gets up to Xaml-status (plus it suffers from multiple-artifact syndrome which I still think needs to be resolved elegantly). I also feel that Xaml is a natural successor to the existing XML file format, especially since we are talking about a C#/VB/.NET where all of these projects have been built by MSBuild for over a decade (and again is benefiting from the same environment that currently already offers so much tooling support). Admittedly, I am not aware of the other technologies you (or @RichiCoder1) reference, so I guess I have some research to do. I just cringe whenever I see code that can be defined declaratively, and that's really what we're after here. I hope this clarifies my position. Thank you again for your input. |
|
Just thinking about this some more and to echo what @RichiCoder1 said... if you cleanly expose the API as C# (or even .NET) objects, then we Xamlites (I just made that up) can build Xaml around that. It doesn't really need to be anything special, especially with what Xamarin is cooking up these days. There's no reason not to support both JSON and Xaml, I guess is what I am trying to say. Heck, if you make JSON the official supported file type (not saying that you should 😅, but if you do), I could see myself making a fork that enables the corresponding Xaml equivalent functionality, for starters. (FWIW, this is what I intend on doing with ASP.NET 5's new configuration API, which is designed to support different configuration providers/formats.) As long as the MSBuild team is aware of the want/need for multiple declarative file format support and provide the necessary design/hooks to easily consume their API in whatever format we prefer, that's what's really important here. 👍 |
|
I like the XAML idea. I do think that the class in this case is an artifact. However, it is an artifact that also represents the internal .NET object. The System.Xaml.XamlReader class has a mature implementation that reads that in for us, with strong type checking. Because of this, the MsBuild system could pre-compile the build definition into BAML (the binary representation of XAML). This could make the build system much faster, since BAML translates to an in-memory object more efficiently than runtime string parsing and schema checking. |
|
@eatdrinksleepcode is on the right track.+1 to C# scripting |
|
@Michael-DST An pluggable, language/format-agnostic API/interface for MSBuild would probably tops really, as this thread shows theres many different formats that people'd like to define builds in. That being said, that'd be no small task, and still probably require one or more built in build configuration parsers. |
|
Yes, and now thinking about it, there really isn't a need for a proprietary file format (.dast) after all (although you could still create one from a branding/optimized/value perspective). It really comes down to the data (saved/stored format) and the view (presentation format). Roslyn offers pretty exciting improvements in this regard and can handle everything in between. Once Visual Studio loads the file, it can run its own conversion process (via Roslyn) between the two formats. It will be an awesome day in technology when one can open up a .json file, have it render as .xaml via Roslyn conversion, edit/design as .xaml, and then when saved, it saves it as back as .json. And vice versa. :) (Although, JSON does not support markup extensions -- ahem 😇 -- so some additional thought would be required around that and other capability discrepancies.) As an aside, I would be remiss if I didn't mention my vote on Visual Studio for .ast code views. I am beginning to feel that the .dast we are talking about here is an extension component of this concept. You are right in saying that this is no small task, and I hope to inspire some light bulbs out there by mentioning it to get the ball rolling. I feel this thread really underscores that we are "pawing at an elephant in a dark room" here that and this "elephant" is bound to be a legitimate GitHub project at some point -- either officially by MSFT or by someone with extra time on their hands (which is obviously not me at the moment 😄). If someone started it, I for one would totally support it. This is a well-overdue feature for software development! |
|
If it's simply a serialization format such as JSON or YAML, please make the serializer itself that's being used swappable through configuration, etc. Think about that for a second :) Configurable configuration ha. I also like the idea of being able to use C# directly, however I could see that making things a bit more difficult as the abstraction would have to be higher up than just the serializer. |
|
I need help |
|
@Gum-Joe, can you please create a gitter channel for your repo buildup, to further this discussion? I have mentioned our discussion on CoreCLR channel: https://gitter.im/dotnet/coreclr. :) Also slightly related stuff on gyp inspired concepts: aspnet/dnx#3173. The bonus point is in package.json, we get intellisense for free even in Community Edition of VS2015! The intellisense goes like this:
Some of these features will come from this VS extension: https://github.com/madskristensen/NpmTaskRunner, which along with https://github.com/madskristensen/WebEssentials2015 are prototype projects (a play ground) for VS vNext features (Mads is the manager of VS web tooling webdev team). There is a lot that can be done by mix and match. @Michael-DST, regarding POCO mapping, I think WE2015 uses Roslyn to map code layout between C# and TypeScript to provide the features like "right-click C# file in Solution Explorer, hover over Web Essentials and click Generate TypeScript interface". There are other extensions to translate the whole .NET dll to form TS interface typings. The similar approaches can be used to reverse transform and generate POCO / DTOs types in C#. We can even run |
|
@jasonwilliams200OK that is great from a new MSFT/web developer/TS perspective, but again it seems to lack any consideration and/or thought from a veteran/MSFT/C#/.NET perspective, hence the described friction/growing frustration from the veteran crowd. What the veteran .NET/MSFT crowd wants is to work in .NET/C#/Xaml and build their solutions as such -- NOT TypeScript/JSON. Additionally, it seems as if you are confirming my suspicions that the package.json file is not made from a serialized .NET POCOs (lower TCO) but is meant to be loaded as a dynamic/schemaless object which is more expensive (higher TCO) to work with, as the schema has to be inferred from a loosely/dynamic-typed mechanism. Please correct me if I am wrong here and/or if I am completely misunderstanding something. It wouldn't be the first time this has happened. :) |
|
@jasonwilliams200OK: I have created a gitter chat at https://gitter.im/Gum-Joe/buildup |
|
@Michael-DST, I totally understand the rationale of sticking to MSBuild from veteran C# devs' perspective. I was just highlighting the fact that today with the virtue of MSBuild extensibility, we can work with those facets and offload many build related stuff to this JSON-driven manifest. Ultimately, my hope is MSBuild to provide JSON and YAML schemas to provide parallel support so we can write our build story in the data representation language of our choice. :). |
|
OK great @jasonwilliams200OK. Thank you for the engaged and thoughtful dialogue. I am glad you hear the "other" side and I hope we can all land on a great solution that satisfies each side's (or role's/actor's/persona's) requirements. |
|
Thanks everyone for the feedback, discussion, and insights. We’re strongly considering more concise and expressive markup languages for future build engines. For MSBuild itself, the existing tooling ecosystem (Visual Studio and third-party) is too tied down to the existing XML format. We feel that the risks involved with a change in build-specification representation outweigh the benefits. |
|
@rainersigwald thank you for adding and contribuiting to the discussion. :) I was worried that no one was really paying attention to this, as dialogue from the MSBuild team has been sparse. Am I understanding correctly that you will be sticking to the current XML paradigm/system and no modifications will be planned or made around this? |
|
@Michael-DST That's correct. We're happy to get proposals for (backward-compatible!) changes to project representation, but MSBuild's file format will remain XML (even if we wouldn't choose that if we were starting from scratch today--and I wouldn't!). |
|
@rainersigwald Well that certainly is disappointing. When you say you've considered the risks, I hope you have considered the risk in the form defections from MSBuild altogether for sticking with a format that is pushing (if not pushed over) a decade in age. It does seem there is overwhelming demand and feedback for a switch in (or support for other) file format(s) and it is disappointing to say the least that the MSBuild team will not consider this for the foreseeable future. At the very least this will reflect poorly on the MSBuild product line as the XML format will continue to be difficult to work with and will give it a dated/older, antiquated look and feel. This makes it difficult to build upon and hampers its competitive appeal. In addition to this, I guess the concern is that this seems like such a fundamental concern/issue and it is not getting the urgency/priority from the team. I guess it would help to know what issues you have decided to work on instead to help better understand. Of course, this is just coming from my (opinionated smile) perspective. I could be wrong and completely missing the pulse here. Wouldn't be the first time. :P Thank you for any consideration and continued thoughts towards/around this. |
|
@rainersigwald I guess that is a "yes we considered it and we don't care." :P Thinking that this should be called MSBuild vSame rather than vNext. :P I would like to take you up on your proposals fro backward-compatible project representation, however. You can definitely represent anything in Xaml, and benefit from the built-in tooling available in Visual Studio due to it. What would be the first step to this? Would you like me to try to build you a Xaml-fied version of a MSBuild "script" and we can take it from there? Trying to clarify what you mean by proposal. :) |
|
@Michael-DST It might be helpful to think of this repo as “MSBuild vSafe”—the focus here is on improving the existing MSBuild for its current users, but not making invasive, large-scale changes. We don’t have anything to announce right now, but you might be interested in some things that have been posted externally. |
|
Haha thank you for your reply @rainersigwald. It is much appreciated. I do have to say that the idea of a team of interns building a build system does not inspire the greatest of confidence. In fact, it is probably this exact approach/methodology/strategy that can be attributed to the reason why MSBuild vSafe is in such the state it is now. :P It also sounds like there is talk about inventing a whole new language, and even referencing TypeScript, which is completely incompatible with .NET. Internal MSFT teams do know about .NET, right? It is hard to tell sometimes as it seems like there is always at least one team trying to skate uphill and create something completely new, different, and completely incompatible with all of the work, effort, and knowledge invested in, well, what currently works. Btw, what I am proposing (using your words/suggestion here) is not a large-scale/invasive change, but a way that improves the current development process which I think we all can agree can stand some (read: a lot of) improvement. MSBuild XML is the worst of the worst and truly gives the product a terrible name and developer experience. Every respectable developer I know cringes at the thought of opening a .csproj file and wading through the mire of a schema-less, arbitrary elements that ultimately (if you're lucky) bind to obscure .NET counterparts somewhere that are nearly impossible to track down. It would seem to me that if you are going to focus on improving the existing MSBuild for current users, that this area might be an area to consider in which to do exactly that. With all of this said it does seem like I should bite my tongue until April before I pass any judgement (too late, I know haha). |
ghost
mentioned this issue
|
This is more relevant than ever! ASP.NET Core project.json is going away and is instead planned to use msbuild. So why not bring the storm here, and demand JSON support for msbuild? |
|
@Ciantic you mean JSON or Xaml. 😉 😉 😉 |
|
Given the new climate around project.json and MSBuild, I've created a new issue around this to further the conversation in a new light: #613 |
|
@Mike-EEE
Um wow. Well I don't want to build my solutions with Xaml actually. I like doing front ends in TypeScript/Angular, and I don't like WPF. Wow, so I must not be a veteran developer. My 21 years working development experience on the MS stack, developing COM, ATL, MFC, Win32, .NET from 1.0 beta onwards, experience writing thick client products, thin client apps, monster admin portals, websites. Writing Reactive Extensions systems, highly concurrent systems, trading engines, high performance trade costing systems, all that, and I'm still not a veteran. All I need to do is say that I prefer Xaml to JSON and then I'm a veteran developer. Because veteran .NET developers only use One True Way™, they don't pick and choose the best that they've found from a wide and deep experience. They only use C# and Xaml. Sorry, but that's the same basic argument as the rest of the people on this thread who appear to believe that if you don't like MSBuild it's just because you don't understand it, and it can't be that you used it for years, and rejected it. |
|
PS. I would rather use a non-MSBuild based Xaml build system than a MSBuild based JSON build system. |
[mono] Add Microsoft.NET.Build.Extensions for ns2.0 support Prompted by dotnet/sdk#1294 nuget pkg version: 2.0.0-preview3-20170622-1
and others
Opening this issue to get the discussion going -- xml is very verbose. Many project nowadays opt for configuring using YAML or json.
It would be great to have support for those in msbuild.
The text was updated successfully, but these errors were encountered: