Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce precise PowerShell module version matches #2011

Closed
AnatoliB opened this issue Aug 12, 2021 · 2 comments
Closed

Enforce precise PowerShell module version matches #2011

AnatoliB opened this issue Aug 12, 2021 · 2 comments

Comments

@AnatoliB
Copy link

AnatoliB commented Aug 12, 2021

When an exact PowerShell module version (not a wildcard) is specified in requirements.psd1, make the PowerShell app fail with an explanatory error message and instructions if the version does not match the version on the PowerShell Gallery precisely.

Motivation

PowerShell Functions users specify required module versions in the requirements.psd1 file, for example:

@{
    'Az' = '6.*'
    'PnP.PowerShell' = '1.5'
}

In order to install modules from the PowerShell Gallery, PowerShell worker relies on the Save-Module command supplied with PowerShell, for example:

Save-Module -Name 'PnP.PowerShell' -RequiredVersion '1.5' ...

This command interacts with the PowerShell Gallery and retrieves the requested module content. The Save-Module command is "smart" enough to tolerate certain differences between the version specified in the RequiredVersion parameter and versions actually present on the PowerShell Gallery. For example, there is no version 1.5 for module PnP.PowerShell, but there is version 1.5.0, so this is what Save-Module installs.

In many contexts, this behavior is convenient and appreciated. However, the Managed Dependencies feature implementation also needs a way to determine whether the specified module version is already available on the worker's file system or not, in order to avoid unnecessary repeated downloads. The logic implemented in the PowerShell worker do not match the rules implemented in Save-Module, and it performs the exact string comparison instead. As a result, the worker will successfully install version 1.5.0 instead of 1.5, but the next time it checks whether version 1.5 is installed or not, it will not be able to find it, so it will initiate the download again. This effectively causes each PowerShell worker to install its own copy of modules on every worker start, which significantly hurts cold start and wastes file share space.

The proposal will mitigate the issue by checking the actual module version, comparing it with the version specified by the user, and failing the app if they do not match exactly. This will force the user to specify exact module versions, matching the content actually present on the PowerShell Gallery.

See also: Azure/azure-functions-powershell-worker#647

Impact

We do not have statistics at this point. Gathering this data is possible, but it will take time, as it will require adding logs to the PowerShell worker and deploying it to production. Based on the number of incidents, this is not something happening very often. However, when users do run into this, the impact is unexpected and hard to self-diagnose.

Compat-mode support

Introducing a feature flag is feasible, but the value of this is questionable: fixing the module versions in requirements.psd1 is very easy, and it gives much better results.

Alternatives

As an alternative, we considered making the PowerShell worker mirror the version-matching logic implemented in Save-Module, so, for example, it will consider version 1.5.0 equivalent to 1.5 as well. This would not require users migrating to Functions v4 to change their apps, so this would not be a breaking change.

However, this approach has problems:

  • The version-matching logic in Save-Module may be more complicated than just tolerating zeroes in the last version component.
  • More importantly, this logic is not under our control. It may change with any PowerShell version upgrade, and keeping the PowerShell worker code always in sync would be challenging.

Detection

We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.

Support

We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.

Documentation

The change needs to be documented.

Components impacted

Impacted components:

  • Azure Functions PowerShell Worker

Performance

No performance impact expected.

@Azure Azure locked and limited conversation to collaborators Aug 12, 2021
@AnatoliB
Copy link
Author

Discussion: #2012

@brettsam
Copy link
Member

brettsam commented Sep 2, 2021

Not considering this as a breaking change for v4. An alternative was implemented.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants