Align SDK-Style and MSBuild-Style Builds By Default

[bgribaudo]

Preflight Checklist

• I have installed the latest version of Power Query SDK.
• I have checked existing resources, including the common issues and the release notes.
• I have searched for similar issues.

Problem

The current auto-generated .proj (MSBuild) file specifies a hard coded list of files to include in the connector .mez. This list is not automatically updated as files are added or removed from the extension.

In contrast, the SDK build process uses a wildcard approach to identify the files to include in its build, which results in it dynamically including new files and no longer including deleted files.

Desired Solution

For symmetry and simplicity, could the MSBuild/.proj-style build default to working exactly the same as SDK-style builds?

Alternatives and Workarounds

No response

Additional Context

This could almost be achieved by changing the default .proj file template from containing along the lines of:

<ItemGroup>
  <MezContent Include="TestConn.pq" />
  <MezContent Include="TestConn16.png" />
  <MezContent Include="TestConn20.png" />
  <MezContent Include="TestConn24.png" />
  <MezContent Include="TestConn32.png" />
  <MezContent Include="TestConn40.png" />
  <MezContent Include="TestConn48.png" />
  <MezContent Include="TestConn64.png" />
  <MezContent Include="TestConn80.png" />
  <MezContent Include="resources.resx" />
</ItemGroup>

to

<ItemGroup>
  <MezContent Include="*" Exclude="*.test.pq" />
</ItemGroup>

A catch with the above is that is allows the possibility for multiple .pq files to be included in the .mez, something that MakePQX.exe compile disallows.

Idea: Create a true custom C# build task for packaging .mez files. Internally, this task would call MakePQX.exe compile (or its underlying DLL). This way, MSBuild-style and SDK-style builds will go through exactly the same core build/validation process.

[mattmasson]

I suspect we'll need to create a custom build task in the future to:

1. Perform syntax validation as part of the build (i.e. fail the build process if your connector doesn't "compile")
2. Create a signed .pqx file

I'm not sure we need it just to create the mez file, though. We should be able to align behaviors by changing the default project file template to something that:

• Includes all files in the root workspace directory
• Excludes *.proj, *.test.pq, and *.query.pq files
• Adds a validation step that confirms @(MezContent) contains a single .pq file

Although I'm a little unsure about including all files at the root... you should see all sorts of extra/unexpected files we see in the .mez files people submit for certification.

For what it's worth, internally we tend to use the following pattern for connector projects:

<ItemGroup>
    <MezContent Include="Connector.pq" />
    <MezContent Include="resources.resx" />
    <MezContent Include="*.png" />
    <MezContent Include="*.pqm" />
</ItemGroup>

It's also fairly common for connectors to embed json configuration files, and some follow the pattern of creating separate client_secret and client_id files (although that was originally done for sample purposes).

Tagging this as Help Wanted for now in case someone would like to try out a few things with the default project template to see what works best.

[bgribaudo]

Although I'm a little unsure about including all files at the root... you should see all sorts of extra/unexpected files we see in the .mez files people submit for certification.

Ouch!

If #165 were adopted, do you think this concern would be made moot?