Skip to content

Commit

Permalink
[One .NET] temporary Windows & macOS installers for .NET 6
Browse files Browse the repository at this point in the history
Depends on: dotnet#5195

For .NET 6 Preview 1, we will need to provide our own installers for
the Android workload. This will also unblock the Xamarin.Forms / MAUI
team as they build on top of the iOS and Android workloads for .NET 6.

We will eventually do some kind of "insertion" process to provide our
`.nupkg` files to the dotnet/installer repo, so these installers will
go away completely at some point.

For now, this creates two new installers:

* Microsoft.Android.Workload.msi - installs to `C:\Program Files\dotnet\`
* Microsoft.Android.Workload.pkg - installs to `/usr/local/share/dotnet/`

Both installers have the following file structure underneath the root
directory:

* sdk\5.0.100-rtm.20509.5\EnableWorkloadResolver.sentinel
* sdk-manifests\5.0.100\Microsoft.Android.Workload\**
* packs\Microsoft.Android.Ref\**
* packs\Microsoft.Android.Sdk\**

The installers will have a hard dependency on .NET
5.0.100-rtm.20509.5, so we will need to have clear instructions for
installing the correct version of .NET for the Android workload.

The Windows installer is made using WIX, to mirror what
dotnet/installer is using:

* https://wixtoolset.org/
* https://github.com/dotnet/installer/blob/861a1dd12cb80bd834d0e51442d46ee7d1a4023f/src/redist/targets/GenerateMSIs.targets

The `.msi` will need to be built on Windows and the `.pkg` will need
to be built on macOS. `create-dotnet-msi.csproj` will download the WIX
toolset to `~\android-toolchain\wix` and call `candle.exe` and
`light.exe` appropriately. `create-dotnet-pkg.csproj` is based on
`create-pkg.csproj`, and no additional tooling is needed.

Changes to CI:

* The `create-installers` make target now creates the .NET 6 `.pkg`
  installer. The `mac_build` stage creates one additional installer.
* Any subsequent stages that download the `installers` artifact, now
  use a wildcard for the installer they need. This will improve some
  of the test stages that were downloading both the `.vsix` and `.pkg`
  installers before.
* A `.NET 6 Preview Installers` Github status will appear that
  contains public download links for the new installers.
* A `Build Results - .NET 6 Preview Installers` artifact will contain
  additional build logs.
  • Loading branch information
jonathanpeppers committed Oct 23, 2020
1 parent d60d160 commit c2c76cb
Show file tree
Hide file tree
Showing 18 changed files with 612 additions and 34 deletions.
1 change: 1 addition & 0 deletions Configuration.props
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
<!-- Version number from: https://github.com/dotnet/installer#installers-and-binaries -->
<DotNetPreviewVersionBand Condition=" '$(DotNetPreviewVersionBand)' == '' ">5.0.100</DotNetPreviewVersionBand>
<DotNetPreviewVersionFull Condition=" '$(DotNetPreviewVersionFull)' == '' ">$(DotNetPreviewVersionBand)-rtm.20509.5</DotNetPreviewVersionFull>
<WixToolPath Condition=" '$(WixToolPath)' == '' ">$(AndroidToolchainDirectory)\wix\</WixToolPath>
<AndroidCmakeVersion Condition=" '$(AndroidCmakeVersion)' == '' ">3.10.2</AndroidCmakeVersion>
<AndroidCmakeVersionPath Condition=" '$(AndroidCmakeVersionPath)' == '' ">$(AndroidCmakeVersion).4988404</AndroidCmakeVersionPath>
<AndroidSdkCmakeDirectory>$(AndroidSdkDirectory)\cmake\$(AndroidCmakeVersionPath)</AndroidSdkCmakeDirectory>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
using System;
using System.IO;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace Xamarin.Android.Tools.BootstrapTasks
{
/// <summary>
/// Used for converting plain text license files to .rtf format.
/// Assumes the file is line wrapped with Environment.NewLine:
/// * Double new lines are preserved.
/// * Single new lines are replaced with spaces.
///
/// For a Unicode escape the control word \u is used, followed by
/// a 16-bit signed decimal integer giving the Unicode UTF-16 code
/// unit number. More information under 'Character encoding' here:
/// https://en.wikipedia.org/wiki/Rich_Text_Format
/// </summary>
public class ConvertToRichText : Task
{
[Required]
public string SourceFile { get; set; }

[Required]
public string DestinationFile { get; set; }

public override bool Execute ()
{
var text = File.ReadAllText (SourceFile);

text = text
.Replace (@"\", @"\\")
.Replace ("{", @"\{")
.Replace ("}", @"\}")
// Only want to keep "double" new lines
.Replace (Environment.NewLine + Environment.NewLine, $@"\par{Environment.NewLine} \par{Environment.NewLine} ")
.Replace (Environment.NewLine, " ");

Directory.CreateDirectory (Path.GetDirectoryName (DestinationFile));
using (var writer = File.CreateText (DestinationFile)) {
writer.Write (@"{\rtf1\ansi\ansicpg1250\deff0{\fonttbl\f0\fcharset0 Courier New;}\f0\pard ");
foreach (char letter in text) {
if (letter <= 0x7f) {
writer.Write (letter);
} else {
writer.Write ("\\u");
writer.Write (Convert.ToUInt32 (letter));
writer.Write ("?");
}
}
writer.Write (" } ");
}

return !Log.HasLoggedErrors;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Xml;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace Xamarin.Android.Tools.BootstrapTasks
{
/// <summary>
/// Generates a .wix file for the contents of ~/android-toolchain/dotnet/packs
/// The .wix file can be used to generate the .msi installer for Windows.
/// </summary>
public class GenerateWixFile : Task
{
[Required]
public string Template { get; set; }

[Required]
public string DestinationFile { get; set; }

[Required]
public string DotNetPath { get; set; }

[Required]
public string DotNetVersion { get; set; }

[Required]
public string MSIVersion { get; set; }

public override bool Execute ()
{
var settings = new XmlWriterSettings {
OmitXmlDeclaration = true,
Indent = true,
};

var directories = new StringBuilder ();
var components = new StringBuilder ();
using (var packWriter = XmlWriter.Create (directories, settings))
using (var componentWriter = XmlWriter.Create (components, settings)) {

// Components
componentWriter.WriteStartElement ("ComponentGroup");
componentWriter.WriteAttributeString ("Id", "ProductComponents");
componentWriter.WriteStartElement ("ComponentRef");
componentWriter.WriteAttributeString ("Id", "EnableWorkloadResolver");
componentWriter.WriteEndElement (); // </ComponentRef>

// dotnet
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "dotnet");
packWriter.WriteAttributeString ("Name", "dotnet");

// sdk
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "sdk");
packWriter.WriteAttributeString ("Name", "sdk");

// DOTNETVERSION
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "DOTNETVERSION");
packWriter.WriteAttributeString ("Name", DotNetVersion);
packWriter.WriteAttributeString ("FileSource", Path.Combine (DotNetPath, "sdk", DotNetVersion));

// EnableWorkloadResolver
packWriter.WriteStartElement ("Component");
packWriter.WriteAttributeString ("Id", "EnableWorkloadResolver");
packWriter.WriteStartElement ("File");
packWriter.WriteAttributeString ("Id", "EnableWorkloadResolver");
packWriter.WriteAttributeString ("Name", "EnableWorkloadResolver.sentinel");
packWriter.WriteAttributeString ("KeyPath", "yes");
packWriter.WriteEndElement (); // </File>
packWriter.WriteEndElement (); // </Component>
packWriter.WriteEndElement (); // </Directory> DOTNETVERSION
packWriter.WriteEndElement (); // </Directory> sdk

// sdk-manifests
var sdk_manifests_root = Path.Combine (DotNetPath, "sdk-manifests");
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "sdk_manifests");
packWriter.WriteAttributeString ("Name", "sdk-manifests");

// 5.0.100
var sdk_manifests = Directory.EnumerateDirectories (sdk_manifests_root).FirstOrDefault ();
if (string.IsNullOrEmpty (sdk_manifests)) {
Log.LogError ($"Cannot find child directory of: {sdk_manifests_root}");
return false;
}
var version_band = Path.GetFileName (sdk_manifests);
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "DOTNETVERSIONBAND");
packWriter.WriteAttributeString ("Name", version_band);
packWriter.WriteAttributeString ("FileSource", sdk_manifests);
var workload = Path.Combine (sdk_manifests, "Microsoft.NET.Workload.Android");
if (Directory.Exists (workload)) {
RecurseDirectory (sdk_manifests, packWriter, componentWriter, workload);
} else {
Log.LogError ($"Cannot find directory: {workload}");
return false;
}
packWriter.WriteEndElement (); // </Directory> version_band
packWriter.WriteEndElement (); // </Directory> sdk-manifests

// packs
var packs_dir = Path.Combine (DotNetPath, "packs");
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", "packs");
packWriter.WriteAttributeString ("Name", "packs");
foreach (var directory in Directory.EnumerateDirectories (packs_dir, "Microsoft.Android.*")) {
RecurseDirectory (packs_dir, packWriter, componentWriter, directory);
}

packWriter.WriteEndDocument (); // </Directory>
componentWriter.WriteEndDocument (); // </ComponentGroup>
}

var template = File.ReadAllText (Template);
var contents = template
.Replace ("@MSIVERSION@", MSIVersion)
.Replace ("@DIRECTORIES@", directories.ToString ())
.Replace ("@COMPONENTS@", components.ToString ());

Log.LogMessage (MessageImportance.Low, "Writing XML to {0}: {1}", DestinationFile, contents);
File.WriteAllText (DestinationFile, contents);

return !Log.HasLoggedErrors;
}

static void RecurseDirectory (string top_dir, XmlWriter packWriter, XmlWriter componentWriter, string directory)
{
var directoryId = GetId (top_dir, directory);
packWriter.WriteStartElement ("Directory");
packWriter.WriteAttributeString ("Id", directoryId);
packWriter.WriteAttributeString ("Name", Path.GetFileName (directory));
packWriter.WriteAttributeString ("FileSource", directory);
foreach (var child in Directory.EnumerateDirectories (directory)) {
var directoryName = Path.GetFileName (child);
if (directoryName.StartsWith (".") || directoryName.StartsWith ("_"))
continue;
RecurseDirectory (top_dir, packWriter, componentWriter, child);
}
foreach (var file in Directory.EnumerateFiles (directory)) {
var fileName = Path.GetFileName (file);
if (fileName.StartsWith (".") || fileName.StartsWith ("_"))
continue;
var componentId = GetId (top_dir, file);
packWriter.WriteStartElement ("Component");
packWriter.WriteAttributeString ("Id", componentId);
packWriter.WriteStartElement ("File");
packWriter.WriteAttributeString ("Id", componentId);
packWriter.WriteAttributeString ("Name", Path.GetFileName (file));
packWriter.WriteAttributeString ("KeyPath", "yes");
packWriter.WriteEndElement (); // </File>
packWriter.WriteEndElement (); // </Component>
componentWriter.WriteStartElement ("ComponentRef");
componentWriter.WriteAttributeString ("Id", componentId);
componentWriter.WriteEndElement (); // </ComponentRef>
}
packWriter.WriteEndElement (); // </Directory>
}

static string GetId (string top_dir, string path)
{
if (string.IsNullOrEmpty (path))
return path;
if (path.Length > top_dir.Length + 1) {
path = path.Substring (top_dir.Length + 1);
}
return GetHashString (path);
}

static byte [] GetHash (string inputString)
{
using (var algorithm = SHA256.Create ())
return algorithm.ComputeHash (Encoding.UTF8.GetBytes (inputString));
}

static string GetHashString (string inputString)
{
var sb = new StringBuilder ("S", 65);
foreach (byte b in GetHash (inputString))
sb.Append (b.ToString ("X2"));
return sb.ToString ();
}
}
}
4 changes: 2 additions & 2 deletions build-tools/automation/azure-pipelines-oss.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ stages:
displayName: make jenkins
- script: >
echo "make create-installers V=1 CONFIGURATION=$(XA.Build.Configuration)" &&
make create-installers V=1 CONFIGURATION=$(XA.Build.Configuration)
echo "make create-pkg create-vsix V=1 CONFIGURATION=$(XA.Build.Configuration)" &&
make create-pkg create-vsix V=1 CONFIGURATION=$(XA.Build.Configuration)
workingDirectory: $(Build.SourcesDirectory)
displayName: create installers
Expand Down
71 changes: 64 additions & 7 deletions build-tools/automation/azure-pipelines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,13 @@ stages:
artifactName: $(TestAssembliesArtifactName)
targetPath: xamarin-android/bin/Test$(XA.Build.Configuration)

- task: MSBuild@1
displayName: pack all nupkgs
inputs:
solution: $(System.DefaultWorkingDirectory)/xamarin-android/build-tools/create-packs/Microsoft.Android.Sdk.proj
configuration: $(XA.Build.Configuration)
msbuildArguments: /t:CreateAllPacks /restore /p:NuGetLicense=$(System.DefaultWorkingDirectory)/xamarin-android/external/monodroid/tools/scripts/License.txt /bl:$(System.DefaultWorkingDirectory)/xamarin-android/bin/Build$(XA.Build.Configuration)/create-all-packs.binlog

# Create installers
- script: make create-installers V=1 CONFIGURATION=$(XA.Build.Configuration)
workingDirectory: $(System.DefaultWorkingDirectory)/xamarin-android
Expand All @@ -227,13 +234,6 @@ stages:
artifactName: $(InstallerArtifactName)
targetPath: xamarin-android/bin/Build$(XA.Build.Configuration)/$(InstallerArtifactName)

- task: MSBuild@1
displayName: pack all nupkgs
inputs:
solution: $(System.DefaultWorkingDirectory)/xamarin-android/build-tools/create-packs/Microsoft.Android.Sdk.proj
configuration: $(XA.Build.Configuration)
msbuildArguments: /t:CreateAllPacks /restore /p:NuGetLicense=$(System.DefaultWorkingDirectory)/xamarin-android/external/monodroid/tools/scripts/License.txt /bl:$(System.DefaultWorkingDirectory)/xamarin-android/bin/Build$(XA.Build.Configuration)/create-all-packs.binlog

- task: NuGetCommand@2
displayName: push nupkgs
inputs:
Expand Down Expand Up @@ -1115,6 +1115,60 @@ stages:

- template: yaml-templates/fail-on-issue.yaml

- stage: dotnet_installers
displayName: .NET 6 Preview Installers
dependsOn: mac_build
jobs:
# Check - "Xamarin.Android (.NET 6 Preview Installers Create .msi and Upload)"
- job: dotnet_preview_installers
displayName: Create .msi and Upload
pool: $(HostedWinVS2019)
workspace:
clean: all
steps:
- checkout: self
submodules: recursive

- task: DownloadPipelineArtifact@2
inputs:
artifactName: $(NuGetArtifactName)
downloadPath: $(System.DefaultWorkingDirectory)\bin\Build$(XA.Build.Configuration)\nupkgs

- task: DownloadPipelineArtifact@2
inputs:
artifactName: $(InstallerArtifactName)
downloadPath: $(System.DefaultWorkingDirectory)\installer-artifacts
patterns: Microsoft.*.pkg

- task: MSBuild@1
displayName: msbuild Xamarin.Android.BootstrapTasks
inputs:
solution: Xamarin.Android.BootstrapTasks.sln
configuration: $(XA.Build.Configuration)
msbuildArguments: /restore /bl:$(System.DefaultWorkingDirectory)\bin\Build$(XA.Build.Configuration)\msbuild-bootstraptasks.binlog

- task: MSBuild@1
displayName: msbuild /t:CreateWorkloadInstallers
inputs:
solution: Xamarin.Android.sln
configuration: $(XA.Build.Configuration)
msbuildArguments: /t:CreateWorkloadInstallers /bl:$(System.DefaultWorkingDirectory)\bin\Build$(XA.Build.Configuration)\msbuild-workload.binlog

- script: copy /Y $(System.DefaultWorkingDirectory)\bin\Build$(XA.Build.Configuration)\*.msi $(System.DefaultWorkingDirectory)\installer-artifacts
displayName: copy .msi

- template: upload-to-storage\win\v1.yml@yaml
parameters:
ArtifactsDirectory: $(System.DefaultWorkingDirectory)\installer-artifacts
Azure.ContainerName: $(Azure.Container.Name)
Azure.BlobPrefix: $(Build.DefinitionName)/public/$(Build.BuildId)/$(Build.SourceBranchName)/$(Build.SourceVersion)
GitHub.Context: .NET 6 Preview Installers

- template: yaml-templates/upload-results.yaml
parameters:
solution: build-tools\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj
artifactName: Build Results - .NET 6 Preview Installers

- stage: finalize_installers
displayName: Finalize Installers
dependsOn: mac_build
Expand Down Expand Up @@ -1148,6 +1202,9 @@ stages:
inputs:
artifactName: $(InstallerArtifactName)
downloadPath: $(System.DefaultWorkingDirectory)/storage-artifacts
patterns: |
xamarin.android*.pkg
Xamarin.Android*.vsix
- powershell: |
$pkg = Get-ChildItem -Path "$(System.DefaultWorkingDirectory)/storage-artifacts/*" -Include *.pkg -File
Expand Down
9 changes: 9 additions & 0 deletions build-tools/automation/yaml-templates/run-installer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ steps:
inputs:
artifactName: $(InstallerArtifactName)
downloadPath: $(System.DefaultWorkingDirectory)
patterns: xamarin.android*.pkg
condition: and(succeeded(), eq(variables['agent.os'], 'Darwin'))

- task: DownloadPipelineArtifact@2
inputs:
artifactName: $(InstallerArtifactName)
downloadPath: $(System.DefaultWorkingDirectory)
patterns: Xamarin.Android*.vsix
condition: and(succeeded(), eq(variables['agent.os'], 'Windows_NT'))

- powershell: |
$itemPattern = "*.vsix"
Expand Down
Loading

0 comments on commit c2c76cb

Please sign in to comment.