[BUG] Azure Function PowerShell: Random "The object is used in the context different from the one associated with the object." errors

[heinrich-ulbricht]

Reporting an Issue or Missing Feature

Issue

Expected behavior

I expect working with Connections returned via -ReturnConnection:true and set via -Connection to work reliably.

Actual behavior

I've got an Azure Function running PowerShell where I use PnP PowerShell extensively. Parallel PowerShell worker processes are possible, if that matters.

In my script I get sporadic errors: System.InvalidOperationException: The object is used in the context different from the one associated with the object. Where the errors happen they don't make sense.

Steps to reproduce behavior

Unfortunately I cannot reproduce this reliably and it's happening in production. Errors go to the log which reliably shows the line number where the error is happening. It might be something mundane like:

Get-PnPListItem -List "ListName" -Connection $Connection -Query "[....]"

The script uses multiple connections but should only have one active at the time the error happens. That makes it even stranger.

So far I have not good theory about what's causing this. It seems random. The Azure function is configured with FUNCTIONS_WORKER_PROCESS_COUNT==2, but processes are isolated, right? If not, parallelism might be a cause.

While searching for solutions I came across those:

• Random issues using PnP Powershell Cmdlets in Azure runbooks when running concurrently PnP-PowerShell#2248
• Set-PnPContext not switching the context! #1729

Maybe this would even be solved by this: #1919 ?

What is the version of the Cmdlet module you are running?

PnP.PowerShell 1.10.0

Which operating system/environment are you running PnP PowerShell on?

• Windows
• Linux
• MacOS
• Azure Cloud Shell
• Azure Functions
• Other : please specify

[gautamdsheth]

Maybe you can use the nightly builds and test that out ?

[heinrich-ulbricht]

@gautamdsheth Will do! But have to wait for #1919 (comment)

[KoenZomers]

@heinrich-ulbricht #1919 has already been merged last week. Please download the latest nightly and upload it to your Azure Function and test it against this. Fair chance this issue will be resolved in the nightly build already.

[heinrich-ulbricht]

@KoenZomers Unfortunately I need #1958 - I will test the nightly as soon as this is merged. Eagerly waiting like on heiße Kohlen.

[KoenZomers]

I'm confused :) #1958 has also been merged last night, so should be in the current prerelease already.

[heinrich-ulbricht]

@KoenZomers At the time of writing the pull request was display as "Open" here, thus my assumption it still has to be merged. It now shows as merged. Strange. But cool! Thank you so much! I think I'll wait for another nightly to be sure and will put it together over the weekend. This is perfect timing since our provisioning solution (that heavily heavily depends on PnP PowerShell) will go into full force the coming week.

Edit: not so strange looking at the history:

:D Or my assumption is wrong as to which branch the nightly build is based on.

[KoenZomers]

Curious to hear about your findings. Sounds like it will be a nice test case for PnP PowerShell to see if everything still works well. The changes I've done through the two PRs mentioned in this thread can have a potential high impact. I think in the end it makes things much more reliable and stable, but it might be a bit bumpy road to get to that point with scripts that are currently running. Let's see how things go.

[heinrich-ulbricht]

@KoenZomers Something is not ok. The script that used to work chokes on the first use of a connection. I pulled the current source from dev, built it, attached a debugger and it fails here:

PnPConnection.Current is null (PnPWebCmdlet.cs). Do you have any immediate ideas?

Connection is done via:

$connection = Connect-PnPOnline -Url $siteUrl -Tenant "contoso.onmicrosoft.com" -ClientId $clientId -Thumbprint $thumbprint -ReturnConnection:$true -ErrorAction Stop

PnP Call that triggers the exception shown in the screenshot:

$items = Get-PnPListItem -List $listName -Fields vdKey,vdValue -Connection $connection

That's it.

PowerShell 7.2
Windows 11

Edit:
Should PnPConnection.Current be set here in ConnectOnline.cs as well in the first case?

Edit2:
Yes, setting PnPConnection.Current there makes my script run again. @KoenZomers Can you confirm that this is the right thing to do? Was the else intentional or should it go and PnPConnection.Current be always set? The surrounding code was authored two years ago by @erwinvanhunen.

[KoenZomers]

Would need to test it with the exact connect you're using. The if has been placed there deliberately. The idea is that if you would use ReturnConnection that it no longer also has an impact on the current connection which you can make without using -ReturnConnection. Why else would you use the parameter anyway if that's the desired result? Only the returned variable would be that connection. It surprises me though that in your sample you do seem to use it correctly. So it can either be something odd with specifically that connection method or with those type of cmdlets.

[heinrich-ulbricht]

With the latest nightly it just fails with the first call to Get-PnPListItem after connecting - which tries to work with PnPConnection.Current which is null.

$connection = Connect-PnPOnline -Url $siteUrl -Tenant "contoso.onmicrosoft.com" -ClientId $clientId -Thumbprint $thumbprint -ReturnConnection:$true -ErrorAction Stop
$items = Get-PnPListItem -List $listName -Fields vdKey,vdValue -Connection $connection

Above code triggers the use of PnPConnection.Current. Or I am making something wrong and can't see it right away.

@KoenZomers Is PnPConnection.Current never supposed to be used when using the connection explicitly via -Connection? Because PnPWebCmdlet certainly is using it.

Edit:
One alternative seems to be this, at least for a successful Get-PnPListItem invocation:

But there is more use of PnPConnection.Current in there:

Edit2:
After staring a bit more at the code I get the impression that all those places working with PnPConnection.Current need to be extended to alternatively work with the provided Connection, if there is any.

Edit3:
And I hope that I am wrong because searching for PnPConnection.Current shows an awful lot of places where it's used. Seems like everything derived from PnPSharePointCmdlet could just use Connection instead?

But now I'm speculating and rather wait for your take on this.

Edit4: (or not)
This looks a lot better and works:

Edit5:
Next bump: PnPGraphCmdlet seems to ignore the Connection pretty well. My workaround for that is to Connect-PnPOnline without returning the connection, giving PnPGraphCmdlet the PnPConnection.Current it needs. But at the same time masking possible problems lurking in the code. PnPGraphCmdlet needs the same revamp you gave to PnPSharePointCmdlet.

[heinrich-ulbricht]

A fixed version is running fine in our environment.

Those fixes were needed:

• PnP.PowerShell: for PnPWebCmdlet replace every PnPConnection.Current with Connection
• PnP.PowerShell: currently this fix for Remove-PnPListItem is needed: 1959
• my PowerShell script: do one Connect-PnPOnline without -ReturnConnection to set PnPConnection.Current so that PnPGraphCmdlet can work

Two "wrong context" errors and connection-related errors that I experienced in the past seem gone since using this new version. One of the "wrong context" errors was in my script, that was missing one -Connection parameter. This kind of error now immediately throws when you don't have a default connection. So I assume some PnP PowerShell users that use explicit connections are up for a surprise as missing Connection parameters are not masked anymore. Since now there really is no default connection.

fyi @KoenZomers @gautamdsheth

[KoenZomers]

@heinrich-ulbricht also thanks for sharing your findings here. Point 1 and 3 sound perfect. Would you mind creating a PR for number 1? Happily merge it.

[KoenZomers]

Never mind @heinrich-ulbricht, adding it to the code already :)

[KoenZomers]

For point 3, trying to see if I can make it so that every PnP cmdlet supports passing in an optional -Connection so things become really flexible and consistent.

[KoenZomers]

I think I have succeeded in tackling point 3 with PR #1982. I would appreciate it if everyone can give the latest nightly an extensive test once this PR has been merged to see if there will not be side effects to this major refactoring.

[heinrich-ulbricht]

@KoenZomers Thank you for the pull request! This was faster and more comprehensive than I could've done it since the project where I need this fixed is consuming all time at the moment. The bug costs time but there is no time to properly fix the bug. As always, the circle. That's also why I won't test the nightly right away in that project, but with the next PowerShell project that comes up, which happens regularly.

Thanks, and wow that is a lot of changes. Hoisting the connection one level up feels right.

[jackpoz]

The unexpected behavior of Disconnect-PnPOnline -Connection has great impact on scripts that work with multiple connections and on hosts that run multiple scripts instances like Azure Automation.
It might be worth to mention this in a more visible way (blog, M365 community calls, you name it) to give it more visibility. I'm sure a lot of developers will want to change their scripts to avoid accidentally closing the wrong connection. (I have warned my colleagues at work rigth away after reading the 1.11 release notes :P )

[KoenZomers]

@jackpoz You're right, this should be given more visibility. Let us discuss options for doing so.

Note that specifically the Disconnect-PnPOnline -Connection was a bug that unfortunately got introduced in 1.11. It has been fixed in the nightly builds again. Also note that even before 1.11 it did not do what you probably thought it would do and even more so, it would likely do what you did not expect it to do, if you get my point :) Meaning it would not do anything with the connection passed in through -Connect as PowerShell passes these in by value instead of reference. So whatever we do to it in our code, will not change a thing to the variable itself. What it did do, was touch the current connection. What it also did do was keep the last connected method and credentials. So Disconnect-PnPOnline literally did not do much. Since 1.11 it at least allows the current connection to be wiped from memory and it allows you to connect again using different credentials and/or an authentication method. Before, this would have required closing your PowerShell screen entirely or starting an entire new Runbook or Azure Function. If you wish a specific connection in a variable to be erased from memory for whatever reason, simply set it to $null yourself. I.e. $myconnection = $null.

[jackpoz]

Also note that even before 1.11 it did not do what you probably thought it would do and even more so, it would likely do what you did not expect it to do, if you get my point :)

This is exactly the part that I was referring too, as it affects any PnP PowerShell version. The instructions at https://pnp.github.io/powershell/cmdlets/Disconnect-PnPOnline.html#description are now very clear about how to disconnect, what's left to do is to update all developers who have used Disconnect-PnPOnline -Connection in the last 5 years with what it actually did, as chances are everyone thought it did affect the last connection 😃

Btw nice job with the connection cleanup work you did 👍