WinRM causes 'vagrant up' to fail with Vagrant 1.8.4/Win 2008 R2 guest #7489
Comments
deancsmith
changed the title
Your custom box ( You can tweak this via If you can get to PowerShell during That said, I think we may be running into the same problem (#7486) – likely during "Sharing cookbooks with VM" in your case. |
|
Can you see if Nick's suggestion works for you please? |
|
FYI I'm having the same issue. The settings on base image before vagrant package looks like this: But vagrant up still fails with the "The WS-Management service cannot process the request. This user is allowed a maximum number of 5 concurrent shells, which has been exceeded. Close existing shells or raise the quota for this user." error... Vagrant version 1.8.3 Host operating system Windows 8.1 Pro Guest operating system Windows Server 2008 R2 |
|
Setting MaxConcurrentUsers to 30 as well made a difference! |
|
@JulianMay: now you're bumping into a different limit, which also appears to have been tightened from the Windows default as shipped. Try Something in your provisioning process is locking these resource limits down to levels that seem inappropriate for your use case. |
|
Yep, figured as much... This is a fresh new installation of Windows 2008 r2, and I'm only using the shell-provisioner to echo "hello world", so i guess this is not a particular use case - Might even be worth updating the winrm-config guide at https://www.vagrantup.com/docs/boxes/base.html ? I'll make a note of winrm configs needed to echo "hello world" via shell provisioner and share it here once I finally get it... |
|
Got it running (shell "hello world"), with these extra configs (Not sure if they are sane): |
|
You can find (presumably sane?) defaults here: I would consider using the default It's not clear to me what is meant by "operation", but I assume it's somewhere on the spectrum between a thread and a (sub-)process. |
|
👍 |
|
Hi All My original settings are below. I do not believe I changed these when I initially created this box so I am not sure how the MaxConcurrentOperationsPerUser got as low as 15 (from the Microsoft-listed default of 1,500). I changed these settings as recommended by @nickwright80 and @JulianMay to the below: I then repackaged my box, ran vagrant up and it worked normally, as it did in Vagrant 1.8.1. Thanks all for your help. @sethvargo, I'm not sure how to handle this generally. If fixes can't be made to make rsync use few enough connections, could Vagrant check these settings and warn (or even override them) during the provisioning process, given that it will likely fail with a nasty error otherwise? |
|
I've been seeing a leak of winrm connections too. I put together a quick and dirty fix for it here: master...ggascoigne:fix_winrm_leak I can't claim that this is thoroughly tested but it's been working pretty well so far and has obviated the need for me to reconfigure all of my Windows box files. |
|
Thanks @ggascoigne. We only have a few Windows boxes, so it's not so bad, but we have actually rolled back to 1.8.1 for the moment anyway. |
|
I am having same error without even sharing any folder on Cygwin! :( vagrant up Host path: /home/vermas/Vagrant_Home/Centos/ |
|
@n1t1nv3rma that looks like a different problem, see #7301 for a tweak that might help you. |
|
I can confirm the issue (v1.8.4, Ubuntu 14.04 x64): This Vagrantfile throws the following error: Downgrade to v1.8.1 solved the issue. |
|
@ggascoigne your fix works for me .. simplest fix IMHO for the interim |
|
Seeing this on Server 2008 boxes that work fine with Vagrant < 1.8.4 as well. @ggascoigne your fix appears to work, have you submitted it as a pull request? |
|
I still get this error on Vagrant 1.8.5 on Windows 10 host building a Windows 7 guest. Is there some configuration I need to do beyond what's out of the box? |
|
Yes, it is still happening for me and, no, I'm not the fixer for this so I can't tell us how to get it working reliably. I may have updated my Vagrant build over the last month and that's why I still see it today. (Because I see that I remark above that the @ggascoigne fix did work for me for whatever I was doing back in August.) Today, I'm running Vagrant 1.8.5 on MacOS. My local files for v1.8.5 have the PR #7712 update and so I'll have to pay more attention to why this issue has returned for a different VM. |
|
One workaround is to have a provisioner execute |
|
I'm sorry, I'm pretty new to Vagrant, and I don't know Ruby either. Are you saying I need to write my own provisione, @mwrock? Or is there some generic way I can do that? |
|
oh my apologies. Its much simpler than creating a provisioner. just include this in your |
|
I tried that @mwrock, and it bombs in the same way when it tries to execute the set-item you gave me. I think that in order for that to work, I'd need to make my own box that has that cooked in. |
|
I’ll report that the line, added to the Vagrantfile, as a shell provisioning step, for my self-made box for Softlayer and Windows Server 2012r2 allows the communicator to execute its registered task. (I’m still looking at downstream provisioning before I say, “It Works For Me!” but, at the moment, this task-quota-limit appears to be the issue and the one-liner steps around it.) What I really would like is a way to check the actual reason for the exception at the point of the $register_task.Run attempt and in the exception’s Catch block to craft an error message that tells the poor Vagrantfile author what to do to allow the intention to successfully complete. I want that because in another 4 weeks for another VM, I will have forgotten all about this great patch we found here and that little helpful advice will remind me. |
|
Yeah the challenge is that that provisioning script needs to run before 5 shells are created. Your luck of that actually happening could depend on the plugins you are using. |
|
Apparently, the real challenge is the Windows TaskFolder GetTask and RegisteredTask Run methods do not provide adequate error exceptions to indicate why a registered task cannot be located or run. We seem to forced to deal with this mysterious Null exception. -- On 23 September 2016 at 09:57:05, Matt Wrock ([email protected]) wrote: Yeah the challenge is that that provisioning script needs to run before 5 shells are created. Your luck of that actually happening could depend on the plugins you are using. — |
|
I wrote a positive remark too quickly. What @mwrock's one-liner does is somehow short-circuit provisioning attempts. Here's what gets reported as INFO: ==> sl-win-esrever: WARNING: The updated configuration might affect the operation of the plugins
==> sl-win-esrever: having a per plugin quota value greater than 100. Verify the configuration of
==> sl-win-esrever: all the registered plugins and change the per plugin quota values for the
==> sl-win-esrever: affected plugins.The provisioner then stops. For the one-line shell provision, well, ok, that might be ok. It only has to run a powershell set-item command and that's it. Maybe that is the end of the chef run. However, for the next :chef_solo provisioner, this is also all that happens. First, it shouldn't re-invoke the shell one-liner and secondly, it should go off and run the actual chef recipes. None of that occurs, chef and Vagrant wrap up and think their job is over. |
|
This is a pain for me, because the latest version I've found that doesn't have this problem is 1.8.1, which doesn't play nicely with the latest version of virtualbox on the Windows 10 machines we have at my work. |
|
this all makes me very sad. I'll have a good look at this over the weekend. let me make sure I understand the scenario here @lonniev. You are targetting a 2008r2 vagrant box and trying to converge chef cookbooks via the chef omnibus plugin. Is there a Vagrantfile gist you can share that repros this? Not sure if your box and cookbooks are publicly available. Even if they are not it may be helpful. |
|
It is messing up three of my Vagrant VMs as well. (I'm pleased to have companions in misery but I trade tears for beers here on sunny Friday afternoon.) The Microsoft literature indicates that this quota setting has no effect until after the WinRM service is restarted with "restart-service winrm". Therefore, I wonder if the workaround ever really did anything. Having run set-item on the target once and then rebooting, I can query WSMan: PS WSMan:\localhost\Shell> dir
WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Shell
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String AllowRemoteShellAccess true
System.String IdleTimeout 7200000
System.String MaxConcurrentUsers 10
System.String MaxShellRunTime 2147483647
System.String MaxProcessesPerShell 0
System.String MaxMemoryPerShellMB 512
System.String MaxShellsPerUser 0The shell provisioner in the Vagrantfile should not be necessary. I remove it. The nasty-gram about the Quota limit goes away and the $register_task.Run exception returns. QED: the shell provisioner never really did anything except short-circuit real provisioning attempts. |
|
The "base box" that I am using is a Softlayer image that is Windows Server 2012r2 with a vagrant user, rsync, and sshd added. It has been a reliable base box for Softlayer until this winrm issue surfaced. The image is available to a Softlayer provider with the parameter: sl.image_guid = "7017d7bc-ac22-4bc2-a58f-567d02a522fe" |
|
I have to break away. Be back later. With a Try/Catch handler in the erb that queries the $registered_task.Name property in the Catch block, this is the latest: ==> sl-tt-sync: Installing Chef 12.4.0
==> sl-tt-sync: Running provisioner: chef_solo...
==> sl-tt-sync: Detected Chef (latest) is already installed
==> sl-tt-sync: Generating chef JSON and uploading...
==> sl-tt-sync: Running chef-solo...
==> sl-tt-sync: WinRM_Elevated_Shell not Runnable.
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.So the $registered_task value is a legitimate RegisteredTask and it is just that Run cannot be called on it, even though the WSMan limits are set to Unlimited 0. |
|
hmm @lonniev I am wondering if you are experiencing a different error alltogether than what was originally discussed in this issue. If you are on 2012r2, the I was curious if a winrm restart was required for the new quota so I simulated the problem by openning up 5 PS Remoting sessions and then I adjusted the quota with no restart and I was immediately able to then open a 6th shell where I had previously gotten the error. At any rate, can you confirm that the error you receive is: Thanks! |
|
In those WSMan settings, the MSDN documentation claims that 0 is used for a Unlimited limit. After values like 100 and 1000 made no difference, I ran the latest attempts with 0. This makes no difference. This issue #7489 may have hijacked for a different error at the same method. If I'm to blame for that, I apologize. Currently, the issue I encounter with trying to get winrm to run the WinRM_Elevated_Shell (that is the script to run chef-solo) is that the Run method on the RegisteredTask throws a Element Not Found exception. The WSManFault does not occur. At the moment, I am looking for reasons why that apparently properly registered task with apparently valid encoded script cannot be run. (I removed the pipe to Out-Null on the RegisterTaskDefinition call to capture the output of the register operation. The returned task content looks valid. I then took the embedded powershell command out of that content and ran it in Powershell window. There the encoded command runs fine. This suggests some kind of permission block on winrm or on the vagrant user running scheduled tasks.) |
fix #7489 preventing winrm connection leakage
|
When do you expect this to hit the prime-time? On Fri, Sep 30, 2016 at 12:11 PM, Chris Roberts [email protected]
Thanks for your time, Aaron LEHMANN "Therefore confess your sins to each other and pray for each other so that |
|
This workaround worked for me. Put it before any other provisioners. (if |
|
i have added the following lines to the vagrant file but i am still seeing the same winrm error Error |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
Vagrant version
1.8.4
Host operating system
OS X 10.11.4
Guest operating system
Windows Server 2008 R2
Vagrantfile
Debug output
https://gist.github.com/deancsmith/13b5368f1a105d2af5b16c4b3f7217f9
Expected behavior
Vagrant should load the box, provision etc.
Everything is running fine in 1.8.1 (and 1.7.x etc. previously).
Actual behavior
Vagrant quits before running Berkshelf and Chef.
I think this is the key exception that is causing the vagrant up to rollback:
Steps to reproduce
References
I think this might be related to this but am not sure:
The text was updated successfully, but these errors were encountered: