Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support releasing and patching Windows apps #2712

Merged
merged 46 commits into from
Jan 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
7795254
feat: support releasing and patching Windows apps
bryanoltman Dec 18, 2024
ce4cf21
Merge branch 'main' into bo/windows
felangel Dec 19, 2024
ce5f509
Improve patching, get exe version from file
bryanoltman Dec 20, 2024
bb1d89a
Merge branch 'bo/windows' of github.com:shorebirdtech/shorebird into …
bryanoltman Dec 20, 2024
9d56c65
Merge branch 'main' into bo/windows
bryanoltman Dec 20, 2024
f983443
Cleanup
bryanoltman Dec 20, 2024
0aa7717
Merge branch 'main' into bo/windows
bryanoltman Dec 30, 2024
7c327fe
Add `shorebird preview` for windows
bryanoltman Dec 30, 2024
3471bd0
Coverage, cleanup
bryanoltman Jan 3, 2025
0799861
Merge branch 'main' into bo/windows
bryanoltman Jan 3, 2025
68f7d84
tests
bryanoltman Jan 3, 2025
c13b7e6
lints
bryanoltman Jan 3, 2025
0159526
tests
bryanoltman Jan 3, 2025
1459ae4
tests
bryanoltman Jan 3, 2025
a8943e9
feat(artifact_proxy): add windows desktop files
bryanoltman Jan 4, 2025
af7e46c
Undo artifact proxy changes
bryanoltman Jan 6, 2025
581cfd7
Test coverage
bryanoltman Jan 6, 2025
8234cba
Test coverage
bryanoltman Jan 6, 2025
4b68a01
Merge branch 'main' into bo/windows
bryanoltman Jan 6, 2025
1da1566
Test coverage
bryanoltman Jan 6, 2025
6e25fd8
Test coverage
bryanoltman Jan 6, 2025
474587f
Test coverage
bryanoltman Jan 6, 2025
3d44ad2
Test coverage
bryanoltman Jan 6, 2025
e476b74
Remove unused vars
bryanoltman Jan 6, 2025
9b3e639
Add patch diffing for windows
bryanoltman Jan 7, 2025
c21eac2
Ignore directories in windows file diffing
bryanoltman Jan 7, 2025
0f48c45
Fix tests
bryanoltman Jan 7, 2025
a04b1d2
Test coverage
bryanoltman Jan 7, 2025
e4139e9
remove unused import
bryanoltman Jan 7, 2025
273ca12
Test coverage
bryanoltman Jan 7, 2025
ceb6bb2
Test coverage
bryanoltman Jan 7, 2025
7fbe37c
Test coverage
bryanoltman Jan 7, 2025
0c44342
Test coverage
bryanoltman Jan 7, 2025
891bbd5
Bump flutter version
bryanoltman Jan 8, 2025
246d818
Merge branch 'main' into bo/windows
bryanoltman Jan 8, 2025
222e233
Test coverage
bryanoltman Jan 8, 2025
02edae1
Test coverage
bryanoltman Jan 8, 2025
b2be080
Test coverage
bryanoltman Jan 8, 2025
a56f2ed
Test coverage
bryanoltman Jan 8, 2025
2bdda13
Extract logic to determine windows release path
bryanoltman Jan 8, 2025
38e4d31
Remove unneeded test file
bryanoltman Jan 8, 2025
79c9b9f
Address TODO
bryanoltman Jan 8, 2025
e58ac3c
exe -> win_archive
bryanoltman Jan 8, 2025
1b4c9ce
Add windows beta warnings
bryanoltman Jan 9, 2025
31b9869
Temporarily disable native patch diffing on windows
bryanoltman Jan 9, 2025
a718d04
Spell my name correctly
bryanoltman Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bin/internal/flutter.version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
77b6dc95d4e1f2de2af3c9241b874f0ede96e1ed
56228c343d6c7fd3e1e548dbb290f9713bb22aa9
1 change: 1 addition & 0 deletions packages/shorebird_cli/bin/shorebird.dart
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ Command: shorebird ${args.join(' ')}
patchExecutableRef,
patchDiffCheckerRef,
platformRef,
powershellRef,
processRef,
pubspecEditorRef,
shorebirdAndroidArtifactsRef,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ export 'android_archive_differ.dart';
export 'apple_archive_differ.dart';
export 'file_set_diff.dart';
export 'plist.dart';
export 'windows_archive_differ.dart';
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import 'package:collection/collection.dart';
import 'package:equatable/equatable.dart';

/// Maps file paths to SHA-256 hash digests.
typedef PathHashes = Map<String, String>;

/// Sets of [PathHashes] that represent changes between two sets of files.
class FileSetDiff {
class FileSetDiff extends Equatable {
/// Creates a [FileSetDiff] showing added, changed, and removed file sets.
FileSetDiff({
const FileSetDiff({
required this.addedPaths,
required this.removedPaths,
required this.changedPaths,
Expand Down Expand Up @@ -72,4 +73,7 @@ class FileSetDiff {
$padding$title:
${paths.sorted().map((p) => '${padding * 2}$p').join('\n')}''';
}

@override
List<Object> get props => [addedPaths, removedPaths, changedPaths];
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import 'dart:io';
import 'dart:typed_data';

/// Utilities for interacting with Windows Portable Executable files.
class PortableExecutable {
/// Zeroes out the timestamps in the provided PE file to enable comparison of
/// binaries with different build times.
///
/// Timestamps are DWORD (4-byte) values at:
/// 1. offset 0x110 in the PE header.
/// 2. offset 0x6e14 (seems to be in section 1, need to figure out a robust
/// way to find this).
///
/// https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#coff-file-header-object-and-image
static Uint8List bytesWithZeroedTimestamps(File file) {
final bytes = file.readAsBytesSync();
final timestampLocations = [0x110, 0x6e14];
for (final location in timestampLocations) {
bytes.setRange(location, location + 4, List.filled(4, 0));
}

return bytes;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
import 'dart:io';
import 'dart:isolate';

import 'package:archive/archive_io.dart';
import 'package:crypto/crypto.dart';
import 'package:path/path.dart' as p;
import 'package:shorebird_cli/src/archive_analysis/archive_analysis.dart';
import 'package:shorebird_cli/src/archive_analysis/archive_differ.dart';
import 'package:shorebird_cli/src/archive_analysis/portable_executable.dart';

/// {@template windows_archive_differ}
/// Finds differences between two Windows app packages.
/// {@endtemplate}
class WindowsArchiveDiffer extends ArchiveDiffer {
/// {@macro windows_archive_differ}
const WindowsArchiveDiffer();

String _hash(List<int> bytes) => sha256.convert(bytes).toString();

bool _isDirectoryPath(String path) {
return path.endsWith('/');
}

@override
bool isAssetFilePath(String filePath) {
// We don't care if an empty directory is added or removed, so ignore paths
// that end with a '/'.
return !_isDirectoryPath(filePath) &&
p.split(filePath).contains('flutter_assets');
}

@override
bool isDartFilePath(String filePath) {
return p.basename(filePath) == 'app.so';
}

@override
bool isNativeFilePath(String filePath) {
// TODO(bryanoltman): reenable this check once we can reliably report
// native diffs
// const nativeFileExtensions = ['.dll', '.exe'];
// return nativeFileExtensions.contains(p.extension(filePath));
return false;
}

@override
Future<FileSetDiff> changedFiles(
String oldArchivePath,
String newArchivePath,
) async {
var oldPathHashes = await fileHashes(File(oldArchivePath));
var newPathHashes = await fileHashes(File(newArchivePath));

oldPathHashes = await _updateHashes(
archivePath: oldArchivePath,
pathHashes: oldPathHashes,
);
newPathHashes = await _updateHashes(
archivePath: newArchivePath,
pathHashes: newPathHashes,
);

return FileSetDiff.fromPathHashes(
oldPathHashes: oldPathHashes,
newPathHashes: newPathHashes,
);
}

/// Removes the timestamps from exe headers
Future<PathHashes> _updateHashes({
required String archivePath,
required PathHashes pathHashes,
}) async {
return Isolate.run(() async {
for (final file in _exeFiles(archivePath)) {
pathHashes[file.name] = await _sanitizedFileHash(file);
}

return pathHashes;
});
}

Future<String> _sanitizedFileHash(ArchiveFile file) async {
final tempDir = Directory.systemTemp.createTempSync();
final outPath = p.join(tempDir.path, file.name);
final outputStream = OutputFileStream(outPath);
file.writeContent(outputStream);
await outputStream.close();

final outFile = File(outPath);
final bytes = PortableExecutable.bytesWithZeroedTimestamps(outFile);
return _hash(bytes);
}

List<ArchiveFile> _exeFiles(String archivePath) {
return ZipDecoder()
.decodeStream(InputFileStream(archivePath))
.files
.where((file) => file.isFile)
.where(
(file) => p.extension(file.name) == '.exe',
)
.toList();
}
}
48 changes: 48 additions & 0 deletions packages/shorebird_cli/lib/src/artifact_builder.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import 'package:mason_logger/mason_logger.dart';
import 'package:meta/meta.dart';
import 'package:path/path.dart' as p;
import 'package:scoped_deps/scoped_deps.dart';
import 'package:shorebird_cli/src/artifact_manager.dart';
import 'package:shorebird_cli/src/logging/logging.dart';
import 'package:shorebird_cli/src/os/operating_system_interface.dart';
import 'package:shorebird_cli/src/platform/platform.dart';
Expand Down Expand Up @@ -547,6 +548,53 @@ Either run `flutter pub get` manually, or follow the steps in ${cannotRunInVSCod
return File(outFilePath);
}

/// Builds a windows app and returns the x64 Release directory
Future<Directory> buildWindowsApp({
String? flavor,
String? target,
List<String> args = const [],
String? base64PublicKey,
DetailProgress? buildProgress,
}) async {
await _runShorebirdBuildCommand(() async {
const executable = 'flutter';
final arguments = [
'build',
'windows',
'--release',
...args,
];

final buildProcess = await process.start(
executable,
arguments,
runInShell: true,
// TODO(bryanoltman): support this
// environment: base64PublicKey?.toPublicKeyEnv(),
bryanoltman marked this conversation as resolved.
Show resolved Hide resolved
);

buildProcess.stdout
.transform(utf8.decoder)
.transform(const LineSplitter())
.listen((line) {
logger.detail(line);
// TODO(bryanoltman): update build progress
});

final stderrLines = await buildProcess.stderr
.transform(utf8.decoder)
.transform(const LineSplitter())
.toList();
final stdErr = stderrLines.join('\n');
final exitCode = await buildProcess.exitCode;
if (exitCode != ExitCode.success.code) {
throw ArtifactBuildException('Failed to build: $stdErr');
}
});

return artifactManager.getWindowsReleaseDirectory();
}

/// Given a log of verbose output from `flutter build ipa`, returns a
/// progress update message to display to the user if the line contains
/// a known progress update step. Returns null (no update) otherwise.
Expand Down
15 changes: 15 additions & 0 deletions packages/shorebird_cli/lib/src/artifact_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,21 @@ class ArtifactManager {
.firstWhereOrNull((directory) => directory.path.endsWith('.app'));
}

/// Returns the build/ subdirectory containing the compiled Windows exe.
Directory getWindowsReleaseDirectory() {
final projectRoot = shorebirdEnv.getShorebirdProjectRoot()!;
return Directory(
p.join(
projectRoot.path,
'build',
'windows',
'x64',
bryanoltman marked this conversation as resolved.
Show resolved Hide resolved
'runner',
'Release',
),
);
}

/// Returns the path to the .ipa file generated by `flutter build ipa`.
///
/// Returns null if:
Expand Down
44 changes: 43 additions & 1 deletion packages/shorebird_cli/lib/src/code_push_client_wrapper.dart
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import 'dart:isolate';
import 'package:archive/archive_io.dart';
import 'package:collection/collection.dart';
import 'package:crypto/crypto.dart';
import 'package:equatable/equatable.dart';
import 'package:io/io.dart' as io;
import 'package:mason_logger/mason_logger.dart';
import 'package:meta/meta.dart';
Expand All @@ -31,7 +32,7 @@ import 'package:shorebird_code_push_client/shorebird_code_push_client.dart';
/// {@template patch_artifact_bundle}
/// Metadata about a patch artifact that we are about to upload.
/// {@endtemplate}
class PatchArtifactBundle {
class PatchArtifactBundle extends Equatable {
/// {@macro patch_artifact_bundle}
const PatchArtifactBundle({
required this.arch,
Expand All @@ -55,6 +56,9 @@ class PatchArtifactBundle {

/// The signature of the artifact hash.
final String? hashSignature;

@override
List<Object?> get props => [arch, path, hash, size, hashSignature];
}

// A reference to a [CodePushClientWrapper] instance.
Expand Down Expand Up @@ -520,6 +524,44 @@ aab artifact already exists, continuing...''',
createArtifactProgress.complete();
}

Future<void> createWindowsReleaseArtifacts({
required String appId,
required int releaseId,
required String projectRoot,
required String releaseZipPath,
}) async {
final createArtifactProgress = logger.progress('Uploading artifacts');

try {
// logger.detail('Uploading artifact for $aabPath');
await codePushClient.createReleaseArtifact(
appId: appId,
releaseId: releaseId,
artifactPath: releaseZipPath,
arch: primaryWindowsReleaseArtifactArch,
platform: ReleasePlatform.windows,
hash:
sha256.convert(await File(releaseZipPath).readAsBytes()).toString(),
canSideload: true,
podfileLockHash: null,
);
} on CodePushConflictException catch (_) {
// Newlines are due to how logger.info interacts with logger.progress.
logger.info(
'''

Windows release (exe) artifact already exists, continuing...''',
);
} catch (error) {
_handleErrorAndExit(
error,
progress: createArtifactProgress,
message: 'Error uploading: $error',
);
}
createArtifactProgress.complete();
}

Future<void> createAndroidArchiveReleaseArtifacts({
required String appId,
required int releaseId,
Expand Down
1 change: 1 addition & 0 deletions packages/shorebird_cli/lib/src/commands/patch/patch.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ export 'ios_patcher.dart';
export 'macos_patcher.dart';
export 'patch_command.dart';
export 'patcher.dart';
export 'windows_patcher.dart';
11 changes: 11 additions & 0 deletions packages/shorebird_cli/lib/src/commands/patch/patch_command.dart
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,10 @@ NOTE: this is ${styleBold.wrap('not')} recommended. Asset changes cannot be incl
logger.warn(macosBetaWarning);
}

if (results.releaseTypes.contains(ReleaseType.windows)) {
logger.warn(windowsBetaWarning);
}

final patcherFutures =
results.releaseTypes.map(_resolvePatcher).map(createPatch);

Expand Down Expand Up @@ -254,6 +258,13 @@ NOTE: this is ${styleBold.wrap('not')} recommended. Asset changes cannot be incl
flavor: flavor,
target: target,
);
case ReleaseType.windows:
return WindowsPatcher(
argResults: results,
argParser: argParser,
flavor: flavor,
target: target,
);
}
}

Expand Down
Loading
Loading