Add experimental support for multiple installed versions #4282

JohnMcPMS · 2024-03-18T18:03:52Z

Movement toward north star #2129

Change

This change introduces a new experimental feature flag called sideBySide that enables multiple instances of an installed package to be found by search results. The CompositeSource will fold these all together into a single package result through their common association with available packages.

Warning

This experimental feature affects the search results that the rest of the Windows Package Manager code relies on. Enabling it may change the behavior of downstream components.

Important

Only the commands in winget.exe have been updated to account for this change. Other interfaces will get reasonable defaults, but may not be able to interact properly in all cases.

Installed identifier change

The most visible effect of this feature being enabled is that the identifier generated for installed items is no longer simply the registry key name or package family name but is instead constructed to make it fully unique. This is mostly relevant to packages without an available counterpart. For packages that are registered with the system through the registry, the identifier is now of the form ARP\User\x64\ProductCode. For MSIX, the form is MSIX\Package_Full_Name. The previous identifiers are still searchable data though, so it is hopefully not more difficult to list installed items.

Installed item correlations

The implementation in CompositeSource tries to separate the steps into:

enumerating the basic results with each having at most one installed item and its correlated available packages.
folding the results together to create the final return value.

In an effort to prevent overly broad correlations from available to installed items, there is an additional attempt to ensure that there are no stronger correlations from installed to available. As an example, if a package is split by version while still sharing a name that is reasonably close, searching for one of the version subsets should not attempt to associate all of the installed items so long as the other versions have stronger correlations.

Package A1 has versions 1.0 - 1.5 with names like A Package (1.0)
Package A2 has versions 2.0 - 2.2 with names like A Package (2.0)
Searching for A2 would correlate with version 1.0 because the normalized names are the same
With the above check, it should not return installed version 1.0 as long as the product code is associated with A1

Once the basic results are discovered, the results are folded together. A comment on that function indicates the methodology and possible future improvements:

// Group results in an attempt to have a single result that covers all installed versions.
// This is expected to be called immediately after the installed search portion,
// when each result will contain a single installed version and some number of available packages.
// 
// The folds that happen are:
//  1. When results have the same primary available package (the primary available package is set due to tracking data)
//  2. When a result has no primary available package, but another result does have a primary that matches one of the available
//      a. Choose the latest primary if there are multiple
//  3. When multiple results have no primary available package and share the same available package set
//      a. There are many potential additional rules that could be made here, but we will start with the simplest version.
//
// Potential improvements:
//  1. Attempting correlation of non-primary available packages to allow folding in more complex cases
//      a. For example, if installed A has {source1:package1, source2:package2} and installed B has {source1:package1}, can we
//          make sure that source1:package1 and source2:package2 are in fact "the same" to confidently say that installed A and B
//          are side by side versions.
//  2. Attempt correlation by installed data only
//      a. We can potentially detect multiple instances of the same installed item with the same correlation logic turned back on
//          the installed source.  This would allow for folding even when the package is not in any available source.

Command changes

list

list has been updated to simply output all versions contained within a single search result as independent lines. This keeps the resulting output basically the same, but that has the downside of not explicitly indicating the correlation between the items.

repair

repair was updated internally so that passing in a --version would cause that specific version to be targeted for repair.

uninstall

uninstall should behave as before if only a single installed version is present. When multiple installed versions exist, an error is produced unless one of two new arguments is supplied:

--all-versions / --all will uninstall all installed versions
--version / -v will uninstall the targeted version only

Discussion

I have created a discussion for this experimental feature: #4281

The goal is to allow feedback to be better collected and threaded, although I expect that there will be issues created independently as well.

Additionally...

I added a script in the tools that launches the sandbox with the dev package registered: Launch-DevPackageInSandbox.ps1

Since it is dev registered in the sandbox, all you need to do to update it is build and deploy from Visual Studio and the sandbox version will just use those latest files.

Validation

Manual validation of the target scenario (multiple installed versions).
Unit tests added for core folding behavior.
More tests should be added before / as part of moving out of experimental.

Microsoft Reviewers: Open in CodeFlow

Trenly · 2024-03-18T18:26:30Z

uninstall

uninstall should behave as before if only a single installed version is present. When multiple installed versions exist, an error is produced unless one of two new arguments is supplied:

--all-versions / --all will uninstall all installed versions

--version / -v will uninstall the targeted version only

Will this also resolve this issue once out of experimental? I only ask because I don't see anywhere in the uninstall flow where --version is being checked, unless I'm missing it in one of the other function calls

Attempting to uninstall a specific version of a package doesn't work #1886

JohnMcPMS · 2024-03-18T20:52:11Z

src/AppInstallerCLICore/Workflows/WorkflowBase.cpp

+                if (context.Args.Contains(Execution::Args::Type::TargetVersion))
+                {
+                    Repository::PackageVersionKey versionKey{ "", context.Args.GetArg(Execution::Args::Type::TargetVersion) , "" };
+                    std::shared_ptr<IPackageVersion> installedVersion = installed->GetVersion(versionKey);
+
+                    if (!installedVersion)
+                    {
+                        context.Reporter.Error() << Resource::String::GetManifestResultVersionNotFound(Utility::LocIndView{ versionKey.Version }) << std::endl;
+                        // This error maintains consistency with passing an available version to commands
+                        AICLI_TERMINATE_CONTEXT(APPINSTALLER_CLI_ERROR_NO_MANIFEST_FOUND);
+                    }
+
+                    context.Add<Execution::Data::InstalledPackageVersion>(std::move(installedVersion));
+                }


@Trenly , selection of target version.

Thanks! I must have missed that

JohnMcPMS · 2024-03-18T20:53:35Z

uninstall

uninstall should behave as before if only a single installed version is present. When multiple installed versions exist, an error is produced unless one of two new arguments is supplied:

--all-versions / --all will uninstall all installed versions

--version / -v will uninstall the targeted version only

Will this also resolve this issue once out of experimental? I only ask because I don't see anywhere in the uninstall flow where --version is being checked, unless I'm missing it in one of the other function calls

Attempting to uninstall a specific version of a package doesn't work #1886

Yes, I believe it would resolve that issue.

yao-msft · 2024-03-21T21:17:00Z

src/AppInstallerCLI.sln

@@ -180,6 +180,11 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "WinGetSourceCreator", "WinG
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.WinGet.SharedLib", "PowerShell\Microsoft.WinGet.SharedLib\Microsoft.WinGet.SharedLib.csproj", "{272B2B0E-40D4-4F0F-B187-519A6EF89B10}"
 EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "schema", "schema", "{46E419FD-C77F-4EFC-A0FC-2BAD93E60D12}"
+	ProjectSection(SolutionItems) = preProject
+		..\schemas\JSON\settings\settings.schema.0.2.json = ..\schemas\JSON\settings\settings.schema.0.2.json


Nit: settings.schema.0.2.json is already listed under WingetSchemas in the solution unless we wanted to explicitly listed here

yao-msft · 2024-03-22T00:04:11Z

src/AppInstallerRepositoryCore/CompositeSource.cpp

@@ -346,45 +343,74 @@ namespace AppInstaller::Repository
        {


nit: the comment "Supports only a single version of a single package at this time." could be removed

yao-msft · 2024-03-22T03:09:58Z

src/AppInstallerRepositoryCore/CompositeSource.cpp

+                        continue;
+                    }
+
+                    // We use the `SourceId` field to store the installed package identifier so that we can disambiguate keys is they have the same version.


Suggested change

// We use the `SourceId` field to store the installed package identifier so that we can disambiguate keys is they have the same version.

// We use the `SourceId` field to store the installed package identifier so that we can disambiguate keys if they have the same version.

yao-msft · 2024-03-22T03:16:21Z

src/AppInstallerRepositoryCore/CompositeSource.cpp

+                    m_versionKeyData.emplace_back(std::move(keyData));
+                }
+
+                m_packages.emplace_back(std::move(package));


In rare cases that all the versions of the package get dropped, do we want to drop the package as well?

yao-msft · 2024-03-22T03:49:54Z

src/AppInstallerRepositoryCore/CompositeSource.cpp

+
+                        for (const auto& availablePackage : currentMatch.Package->GetAvailablePackages())
+                        {
+                            auto& packageFoldData = foldData.at(availablePackage);


Is the availablePackage converted to InstalledResultFoldKey automatically here?

Yes, there is a non-explicit constructor from a std::shared_ptr<IPackage>.

JohnMcPMS added 25 commits January 25, 2024 08:42

Change enumeration to prevent any duplicates from being excluded

1b9954e

Create primary available package in composite

0fc5008

wip

5af5107

WIP; folding after installed

04a638b

Initial folding pass complete, needs to actually fold

349aa61

wip; adding support for multiple installed package entries

a5d262c

pause for integration of pinning changes

29d60fc

Merge from master

8a85d89

Merge to remove conflicts; still needs more concept merging

33a0fba

stash

e93c038

Merge from master with intentional CompositeSource conflicts

c205b45

Manual merge of CompositeSource

c3531fb

CS refactor adaptation in progress

935bdc3

installed folding working again

d8e468c

list shows all installed versions

142556b

spelling and a few comment thoughts

5ec99fa

Refactor installed search/tracking

610e580

Available correlation complete

1b0d8f6

Update command UX and fix bug

6d2371c

list/upgrade/repair/uninstall commands updated

675f3be

minor fixes

407d923

Ignore empty packages

5d45564

Existing unit tests pass

51d7835

Add unit tests

8795687

Merge from master

a32ea1f

JohnMcPMS requested a review from a team as a code owner March 18, 2024 18:03

JohnMcPMS commented Mar 18, 2024

View reviewed changes

Fix tests

25c0151

yao-msft previously approved these changes Mar 22, 2024

View reviewed changes

JohnMcPMS added 3 commits March 22, 2024 10:33

PR feedback

eae34b4

Revert solution changes

fc4598b

Merge branch 'master' into sxs-test

6c0d839

JohnMcPMS dismissed yao-msft’s stale review via 6c0d839 March 22, 2024 18:03

yao-msft approved these changes Mar 22, 2024

View reviewed changes

JohnMcPMS merged commit 8e82bfe into microsoft:master Mar 22, 2024
8 checks passed

JohnMcPMS deleted the sxs-test branch March 22, 2024 23:06

SirusCodes mentioned this pull request Jul 24, 2024

add spec for #1620 #2650

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental support for multiple installed versions #4282

Add experimental support for multiple installed versions #4282

JohnMcPMS commented Mar 18, 2024 •

edited by microsoft-github-policy-service bot

Loading

Trenly commented Mar 18, 2024 •

edited

Loading

uninstall

JohnMcPMS Mar 18, 2024

Trenly Mar 18, 2024

JohnMcPMS commented Mar 18, 2024

uninstall

yao-msft Mar 21, 2024

yao-msft Mar 22, 2024

yao-msft Mar 22, 2024

yao-msft Mar 22, 2024

yao-msft Mar 22, 2024

JohnMcPMS Mar 22, 2024

	// We use the `SourceId` field to store the installed package identifier so that we can disambiguate keys is they have the same version.
	// We use the `SourceId` field to store the installed package identifier so that we can disambiguate keys if they have the same version.

Add experimental support for multiple installed versions #4282

Add experimental support for multiple installed versions #4282

Conversation

JohnMcPMS commented Mar 18, 2024 • edited by microsoft-github-policy-service bot Loading

Change

Installed identifier change

Installed item correlations

Command changes

list

repair

uninstall

Discussion

Additionally...

Validation

Microsoft Reviewers: Open in CodeFlow

Trenly commented Mar 18, 2024 • edited Loading

uninstall

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohnMcPMS commented Mar 18, 2024

uninstall

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohnMcPMS commented Mar 18, 2024 •

edited by microsoft-github-policy-service bot

Loading

Trenly commented Mar 18, 2024 •

edited

Loading