You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When an exact PowerShell module version (not a wildcard) is specified in requirements.psd1, make the PowerShell app fail with an explanatory error message and instructions if the version does not match the version on the PowerShell Gallery precisely.
Motivation
PowerShell Functions users specify required module versions in the requirements.psd1 file, for example:
@{
'Az'='6.*''PnP.PowerShell'='1.5'
}
In order to install modules from the PowerShell Gallery, PowerShell worker relies on the Save-Module command supplied with PowerShell, for example:
This command interacts with the PowerShell Gallery and retrieves the requested module content. The Save-Module command is "smart" enough to tolerate certain differences between the version specified in the RequiredVersion parameter and versions actually present on the PowerShell Gallery. For example, there is no version 1.5 for module PnP.PowerShell, but there is version 1.5.0, so this is what Save-Module installs.
In many contexts, this behavior is convenient and appreciated. However, the Managed Dependencies feature implementation also needs a way to determine whether the specified module version is already available on the worker's file system or not, in order to avoid unnecessary repeated downloads. The logic implemented in the PowerShell worker do not match the rules implemented in Save-Module, and it performs the exact string comparison instead. As a result, the worker will successfully install version 1.5.0 instead of 1.5, but the next time it checks whether version 1.5 is installed or not, it will not be able to find it, so it will initiate the download again. This effectively causes each PowerShell worker to install its own copy of modules on every worker start, which significantly hurts cold start and wastes file share space.
The proposal will mitigate the issue by checking the actual module version, comparing it with the version specified by the user, and failing the app if they do not match exactly. This will force the user to specify exact module versions, matching the content actually present on the PowerShell Gallery.
We do not have statistics at this point. Gathering this data is possible, but it will take time, as it will require adding logs to the PowerShell worker and deploying it to production. Based on the number of incidents, this is not something happening very often. However, when users do run into this, the impact is unexpected and hard to self-diagnose.
Compat-mode support
Introducing a feature flag is feasible, but the value of this is questionable: fixing the module versions in requirements.psd1 is very easy, and it gives much better results.
Alternatives
As an alternative, we considered making the PowerShell worker mirror the version-matching logic implemented in Save-Module, so, for example, it will consider version 1.5.0 equivalent to 1.5 as well. This would not require users migrating to Functions v4 to change their apps, so this would not be a breaking change.
However, this approach has problems:
The version-matching logic in Save-Module may be more complicated than just tolerating zeroes in the last version component.
More importantly, this logic is not under our control. It may change with any PowerShell version upgrade, and keeping the PowerShell worker code always in sync would be challenging.
Detection
We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.
Support
We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.
Documentation
The change needs to be documented.
Components impacted
Impacted components:
Azure Functions PowerShell Worker
Performance
No performance impact expected.
The text was updated successfully, but these errors were encountered:
When an exact PowerShell module version (not a wildcard) is specified in
requirements.psd1
, make the PowerShell app fail with an explanatory error message and instructions if the version does not match the version on the PowerShell Gallery precisely.Motivation
PowerShell Functions users specify required module versions in the
requirements.psd1
file, for example:In order to install modules from the PowerShell Gallery, PowerShell worker relies on the
Save-Module
command supplied with PowerShell, for example:This command interacts with the PowerShell Gallery and retrieves the requested module content. The
Save-Module
command is "smart" enough to tolerate certain differences between the version specified in theRequiredVersion
parameter and versions actually present on the PowerShell Gallery. For example, there is no version1.5
for modulePnP.PowerShell
, but there is version1.5.0
, so this is whatSave-Module
installs.In many contexts, this behavior is convenient and appreciated. However, the Managed Dependencies feature implementation also needs a way to determine whether the specified module version is already available on the worker's file system or not, in order to avoid unnecessary repeated downloads. The logic implemented in the PowerShell worker do not match the rules implemented in
Save-Module
, and it performs the exact string comparison instead. As a result, the worker will successfully install version1.5.0
instead of1.5
, but the next time it checks whether version1.5
is installed or not, it will not be able to find it, so it will initiate the download again. This effectively causes each PowerShell worker to install its own copy of modules on every worker start, which significantly hurts cold start and wastes file share space.The proposal will mitigate the issue by checking the actual module version, comparing it with the version specified by the user, and failing the app if they do not match exactly. This will force the user to specify exact module versions, matching the content actually present on the PowerShell Gallery.
See also: Azure/azure-functions-powershell-worker#647
Impact
We do not have statistics at this point. Gathering this data is possible, but it will take time, as it will require adding logs to the PowerShell worker and deploying it to production. Based on the number of incidents, this is not something happening very often. However, when users do run into this, the impact is unexpected and hard to self-diagnose.
Compat-mode support
Introducing a feature flag is feasible, but the value of this is questionable: fixing the module versions in
requirements.psd1
is very easy, and it gives much better results.Alternatives
As an alternative, we considered making the PowerShell worker mirror the version-matching logic implemented in
Save-Module
, so, for example, it will consider version1.5.0
equivalent to1.5
as well. This would not require users migrating to Functions v4 to change their apps, so this would not be a breaking change.However, this approach has problems:
Save-Module
may be more complicated than just tolerating zeroes in the last version component.Detection
We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.
Support
We will indeed detect that the app is relying on the imprecise version matching, and the error message will be exposed to the user, as this is the entire point of this proposal. The error message should include migration guidance.
Documentation
The change needs to be documented.
Components impacted
Impacted components:
Performance
No performance impact expected.
The text was updated successfully, but these errors were encountered: