NuGet.exe restore does not use basic auth credentials from config

[roryprimrose]

I'm trying to set up a private NuGet feed that uses an API key for pushing packages and a simple basic auth IHttpModule to protect reads. The system fails on the build server when the build attempts to do a NuGet restore from the private feed.

The nuget.exe is invoked with the following powershell:

$ScriptDir = Split-Path -parent $MyInvocation.MyCommand.Path

Write-Host "Restoring NuGet packages"

& "$ScriptDir\nuget.exe" restore "$ScriptDir\..\MySolution.sln" -PackagesDirectory "$ScriptDir\..\Packages" -ConfigFile "$ScriptDir\NuGet.config" -NonInteractive -Verbosity detailed

The referenced config file contains the following (with sensitive info removed):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageRestore>
        <add key="enabled" value="True" />
        <add key="automatic" value="True" />
    </packageRestore>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
  <packageSources>
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
    <add key="Custom" value="https://nuget.myserver.com/nuget/" />
  </packageSources>
  <disabledPackageSources />
  <packageSourceCredentials>
    <Custom>
      <add key="Username" value="[UserName]" />
      <add key="ClearTextPassword" value="[Password]" />
    </Custom>
  </packageSourceCredentials>
</configuration>

The powershell script logs the following for the restore:

Restoring NuGet packages
GET https://www.nuget.org/api/v2/Packages(Id='MySystem.Core',Version='1.0.1')
GET https://www.nuget.org/api/v2/Packages(Id='OtherSystem.Client',Version='2.0.2')
GET https://www.nuget.org/api/v2/Packages(Id='MySystem.Core',Version='1.0.1.0')
GET https://www.nuget.org/api/v2/Packages(Id='OtherSystem.Client',Version='2.0.2.0')
Using credentials from config. UserName: [Username]
GET https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2')
GET https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1')
Installing 'MySystem.Core 1.0.1'.
Installing 'OtherSystem.Client 2.0.2'.
GET https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1
GET https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2
Please provide credentials for: https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2
UserName: Please provide credentials for: https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1
nuget.exe : System.InvalidOperationException: Cannot prompt for input in non-interactive mode.
At C:\Temp\MySystem\Tools\RestorePackages.ps1:5 char:1
+ & "$ScriptDir\nuget.exe" restore "$ScriptDir\..\MySolution.sln" -PackagesDi ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (System.InvalidO...teractive mode.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

   at NuGet.Common.Console.EnsureInteractive()
   at NuGet.Common.Console.ReadLine()
   at NuGet.ConsoleCredentialProvider.GetCredentials(Uri uri, IWebProxy proxy, Creden
tialType credentialType, Boolean retrying)
   at NuGet.SettingsCredentialProvider.GetCredentials(Uri uri, IWebProxy proxy, CredentialType credentialType, Boolean retrying)
   at NuGet.RequestHelper.SetCredentialsOnAuthorizationError(HttpWebRequest reques
t)
   at NuGet.RequestHelper.ConfigureRequest(HttpWebRequest request)
   at NuGet.RequestHelper.GetResponse()
   at NuGet.HttpClient.GetResponse()
   at NuGet.HttpClient.DownloadData(Stream targetStream)
   at NuGet.PackageDownloader.DownloadPackage(I
HttpClient downloadClient, IPackageName package, Stream targetStream)
   at NuGet.PackageDownloader.DownloadPackage(Uri uri, IPackageMetadata package, Stream targetStream)
   at NuGet.DataServicePackage.<EnsurePackage>b__0(Stream stream)
   at NuGet.Mac
hineCache.<>c__DisplayClass32.<InvokeOnPackage>b__31()
   at NuGet.MachineCache.TryAct(Func`1 action, String path)
   at NuGet.MachineCache.InvokeOnPackage(String packageId, SemanticVersion version, Action`1 action)
   at NuGet.DataServicePackage.Ensure
Package(IPackageCacheRepository cacheRepository)
   at NuGet.DataServicePackage.GetFiles()
   at NuGet.PackageManager.ExpandFiles(IPackage package)
   at NuGet.PackageManager.OnExpandFiles(PackageOperationEventArgs e)
   at NuGet.PackageManager.Execute
Install(IPackage package)
   at NuGet.PackageManager.Execute(PackageOperation operation)
   at NuGet.PackageManager.Execute(IPackage package, IPackageOperationResolver resolver)
   at NuGet.PackageManager.InstallPackage(IPackage package, FrameworkName t
argetFramework, Boolean ignoreDependencies, Boolean allowPrereleaseVersions, Boolean ignoreWalkInfo)
   at NuGet.PackageManager.InstallPackage(IPackage package, Boolean ignoreDependencies, Boolean allowPrereleaseVersions, Boolean ignoreWalkInfo)
   at Nu
Get.Common.PackageExtractor.<>c__DisplayClass1.<InstallPackage>b__0()
   at NuGet.Common.PackageExtractor.ExecuteLocked(String name, Action action)
   at NuGet.Common.PackageExtractor.InstallPackage(IPackageManager packageManager, IPackage package)
   a
t NuGet.Commands.RestoreCommand.RestorePackage(IFileSystem packagesFolderFileSystem, String packageId, SemanticVersion version, Boolean packageRestoreConsent, 
ConcurrentQueue`1 satellitePackages)
   at NuGet.Commands.RestoreCommand.<>c__DisplayClass5.<>c_
_DisplayClass7.<ExecuteInParallel>b__2()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
UserName: 

I put some rudimentary logging into the HttpModule and it definitely gets the credentials from config to correctly read the package feed and identify the packages. However it looks like it doesn't use these credentials for actually downloading the package.

I even added in cookie support for caching the basic auth credential in the hope that nuget.exe would use the same CookieContainer but it doesn't.

This is the log from the module:

4/19/2015 7:19:07 AM +00:00: https://nuget.myserver.com/nuget/ - Request for https://nuget.myserver.com/nuget/ - GET
4/19/2015 7:19:07 AM +00:00: https://nuget.myserver.com/nuget/ - No authorization header found
4/19/2015 7:19:07 AM +00:00: https://nuget.myserver.com/nuget/ - Request denied for /nuget/
4/19/2015 7:19:07 AM +00:00: https://nuget.myserver.com/nuget/ - Request for https://nuget.myserver.com/nuget/ - GET
4/19/2015 7:19:08 AM +00:00: https://nuget.myserver.com/nuget/ - No authorization header found
4/19/2015 7:19:08 AM +00:00: https://nuget.myserver.com/nuget/ - Request denied for /nuget/
4/19/2015 7:19:09 AM +00:00: https://nuget.myserver.com/nuget/ - Request for https://nuget.myserver.com/nuget/ - GET
4/19/2015 7:19:10 AM +00:00: https://nuget.myserver.com/nuget/ - Password valid - setting the principal
4/19/2015 7:19:10 AM +00:00: https://nuget.myserver.com/nuget/ - Auth cookie set
4/19/2015 7:19:11 AM +00:00: https://nuget.myserver.com/nuget/ - User authenticated, request is allowed
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request for https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - GET
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - Request for https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - GET
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - No authorization header found
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - Request denied for /nuget/
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - No authorization header found
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - Request for https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - GET
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - Password valid - setting the principal
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - Auth cookie set
4/19/2015 7:19:13 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='MySystem.Core',Version='1.0.1') - User authenticated, request is allowed
4/19/2015 7:19:14 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request denied for /nuget/
4/19/2015 7:19:14 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request for https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - GET
4/19/2015 7:19:14 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Password valid - setting the principal
4/19/2015 7:19:14 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Auth cookie set
4/19/2015 7:19:14 AM +00:00: https://nuget.myserver.com/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - User authenticated, request is allowed
4/19/2015 7:19:18 AM +00:00: https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2 - Request for https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2 - GET
4/19/2015 7:19:18 AM +00:00: https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2 - No authorization header found
4/19/2015 7:19:18 AM +00:00: https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2 - Request denied for /api/
4/19/2015 7:19:19 AM +00:00: https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1 - Request for https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1 - GET
4/19/2015 7:19:19 AM +00:00: https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1 - No authorization header found
4/19/2015 7:19:19 AM +00:00: https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1 - Request denied for /api/

You can see here that for each set of requests, auth fails and then the auth credentials are read from config and provided, the cookie is set and the repeated requests is successful. This continues all the way until nuget.exe attempts to download the packages. At this point the auth credentials from config are not provided which results in a hard fail with no repeat with credentials.

[bhuvak]

@roryprimrose, do you have the credentials set in %AppData%NuGet config file ? Looks like that's the one being picked up while restoring during build

[roryprimrose]

No, I point nuget.exe to a config in the same folder. It is here that the credentials get picked up as seen in the log line "Using credentials from config. UserName: [Username]".

I also tried ditching my config file and using powershell to add the credential via the command line before executing the restore. Exactly the same thing happened though.

[yishaigalatzer]

@roryprimrose are you able to use the credentials from %appdata% Nuget.config as a workaround for now?

[roryprimrose]

See previous two comments. This question has already been addressed.

[yishaigalatzer]

Not really. The question was around if you can use the above suggestion as a workaround? If not we can look for another workaround until this gets fixed and a nuget.exe released

[roryprimrose]

I think I have already covered that and found the same result. I tried using the command line to add the credentials as part of the build. This would put the credentials into %appdata% nuget.config. The creds were used to query the packages but not used to download them.

[yishaigalatzer]

Can you verify the content of your global nuget.config. This is rather odd since this is a mainline scenario and I'm very surprised by it. We can obviously take a look first thing in the morning.

[roryprimrose]

Not really. This is running in a VSO build so it would depend on that config. As you can see in the powershell though, I am getting nuget.exe to point to my specific config.

If it is any help in proving this yourself. I have a web project that has the nuget server package installed and then the following module (supports basic auth with a single config user):

namespace MyServer.NuGet
{
    using System;
    using System.Diagnostics;
    using System.IO;
    using System.Net.Http.Headers;
    using System.Security.Principal;
    using System.Text;
    using System.Threading;
    using System.Web;
    using System.Web.Configuration;
    using System.Web.Security;

    public class BasicAuthenticationModule : IHttpModule
    {
        private const string AuthCookieKey = "2327385720594389A013024C00F09EB6";

        private const string Realm = "NuGet";

        private static readonly object _syncLock = new object();

        public void Dispose()
        {
        }

        public void Init(HttpApplication context)
        {
            // Register event handlers
            context.AuthorizeRequest += OnAuthorizeRequest;
            context.AuthenticateRequest += OnApplicationAuthenticateRequest;
            context.EndRequest += OnApplicationEndRequest;
        }

        private static bool AuthenticateUser(HttpContext context, string credentials)
        {
            try
            {
                var encoding = Encoding.GetEncoding("iso-8859-1");
                credentials = encoding.GetString(Convert.FromBase64String(credentials));

                var separator = credentials.IndexOf(':');
                var name = credentials.Substring(0, separator);
                var password = credentials.Substring(separator + 1);

                if (CheckPassword(name, password))
                {
                    LogMessage("Password valid - setting the principal");

                    SetPrincipal(context, name);

                    return true;
                }
            }
            catch (FormatException)
            {
                // Credentials were not formatted correctly.
            }

            return false;
        }

        private static bool CheckPassword(string username, string password)
        {
            var expectedUserName = WebConfigurationManager.AppSettings["ReaderUserName"];
            var expectedPassword = WebConfigurationManager.AppSettings["ReaderPassword"];

            return username == expectedUserName && password == expectedPassword;
        }

        private static GenericPrincipal CreatePrincipal(string name)
        {
            var identity = new GenericIdentity(name);

            return new GenericPrincipal(identity, null);
        }

        private static IPrincipal GetCookieIdentity(HttpRequest request)
        {
            var authCookie = request.Cookies[AuthCookieKey];

            if (authCookie == null)
            {
                return null;
            }

            var ticket = FormsAuthentication.Decrypt(authCookie.Value);

            if (ticket == null)
            {
                return null;
            }

            if (ticket.Expired)
            {
                LogMessage("Auth cookie expired");

                request.Cookies.Remove(AuthCookieKey);

                return null;
            }

            LogMessage("Auth cookie found");

            var principal = CreatePrincipal(ticket.Name);

            return principal;
        }

        [Conditional("DEBUG")]
        private static void LogMessage(string message)
        {
            var context = HttpContext.Current;
            var path = context.Server.MapPath("~/App_Data") + @"\messages.log";

            lock (_syncLock)
            {
                File.AppendAllText(
                    path, 
                    DateTimeOffset.UtcNow + ": " + context.Request.Url + " - " + message + Environment.NewLine);
            }
        }

        private static void OnApplicationAuthenticateRequest(object sender, EventArgs e)
        {
            var application = (HttpApplication)sender;
            var context = application.Context;
            var request = context.Request;

            LogMessage("Request for " + request.Url + " - " + request.HttpMethod);

            Debug.WriteLine(request.Url + " - " + request.HttpMethod);

            // Check if there is an auth cookie
            var cookieIdentity = GetCookieIdentity(request);

            if (cookieIdentity != null)
            {
                SetPrincipal(cookieIdentity);

                return;
            }

            var authHeader = request.Headers["Authorization"];

            if (authHeader == null)
            {
                LogMessage("No authorization header found");

                return;
            }


            var authHeaderVal = AuthenticationHeaderValue.Parse(authHeader);

            // RFC 2617 sec 1.2, "scheme" name is case-insensitive
            if (authHeaderVal.Scheme.Equals("basic", StringComparison.OrdinalIgnoreCase) &&
                authHeaderVal.Parameter != null)
            {
                var authenticated = AuthenticateUser(context, authHeaderVal.Parameter);

                if (authenticated == false)
                {
                    LogMessage("Authentication failed");

                    context.Response.StatusCode = 401;
                    application.CompleteRequest();
                }
            }
        }

        private static void OnApplicationEndRequest(object sender, EventArgs e)
        {
            // If the request was unauthorized, add the WWW-Authenticate header 
            // to the response.
            var response = HttpContext.Current.Response;
            if (response.StatusCode == 401)
            {
                response.Headers.Add("WWW-Authenticate", string.Format("Basic realm=\"{0}\"", Realm));
            }
        }

        private static void OnAuthorizeRequest(object sender, EventArgs eventArgs)
        {
            var application = (HttpApplication)sender;
            var context = application.Context;
            var requestUri = context.Request.Url.ToString();

            if (Thread.CurrentPrincipal.Identity.IsAuthenticated)
            {
                LogMessage("User authenticated, request is allowed");

                return;
            }

            if (requestUri.IndexOf("/api/", StringComparison.OrdinalIgnoreCase) > -1)
            {
                // If this is a post, allow it as it should have an API key
                if (context.Request.HttpMethod == "PUT")
                {
                    return;
                }

                LogMessage("Request denied for /api/");

                context.Response.StatusCode = 401;
                application.CompleteRequest();
            }
            else if (requestUri.IndexOf("/nuget/", StringComparison.OrdinalIgnoreCase) > -1)
            {
                LogMessage("Request denied for /nuget/");

                context.Response.StatusCode = 401;
                application.CompleteRequest();
            }
            else
            {
                LogMessage("Request allowed by default");
            }
        }

        private static void SetPrincipal(HttpContext context, string name)
        {
            var principal = CreatePrincipal(name);

            const int AuthExpiryInDays = 30;
            var ticket = new FormsAuthenticationTicket(
                name, 
                true, 
                (int)TimeSpan.FromDays(AuthExpiryInDays).TotalMinutes);
            var encyptedTicket = FormsAuthentication.Encrypt(ticket);
            var cookie = new HttpCookie(AuthCookieKey, encyptedTicket)
            {
                Secure = true, 
                HttpOnly = true, 
                Expires = DateTime.UtcNow.AddDays(AuthExpiryInDays)
            };

            LogMessage("Auth cookie set");
            context.Response.Cookies.Add(cookie);

            SetPrincipal(principal);
        }

        private static void SetPrincipal(IPrincipal principal)
        {
            Thread.CurrentPrincipal = principal;
            if (HttpContext.Current != null)
            {
                HttpContext.Current.User = principal;
            }
        }
    }
}

[yishaigalatzer]

Thanks, I was missing the part where you run in vso. That makes sense now.

[bhuvak]

@roryprimrose , I did a setup locally with a nuget.server which has the basic auth module that you have shared and the issue doesn't repro with nuget.exe 2.8.6. However, when proper permissions are not set on the "packages" folder ( for the nuget.server website user account), I hit this issue. In your case does trying to download the package via browser [https://nuget.myserver.com/api/v2/package/mysystem.core/1.0.1] work ?

[roryprimrose]

Ah, interesting. I have pointed the packagesPath config to ~/App_Data/Packages to ensure that the packages are not publicly exposed.

I am surprised that this is a security issue because I expected that the server code would have read access to the packages folder on behalf of the user. What permissions are required outside of any IIS/ASP.Net defaults?

I'll so some more testing around this when I get a little more time.

[roryprimrose]

Just tested this locally using IISExpress. I authenticated via the browser and the auth cookie was set (configured to use ~/App_Data/Packages as well).

I then listed packages and directly downloaded one of the packages using the browser. I got the package and my trace logs also indicated what happened.

3/05/2015 10:12:05 AM +00:00: https://localhost:44390/api/v2/package/othersystem.client/2.0.1 - Request for https://localhost:44390/api/v2/package/othersystem.client/2.0.1 - GET
3/05/2015 10:12:05 AM +00:00: https://localhost:44390/api/v2/package/othersystem.client/2.0.1 - Auth cookie found
3/05/2015 10:12:05 AM +00:00: https://localhost:44390/api/v2/package/othersystem.client/2.0.1 - User authenticated, request is allowed

I then created a restore scenario that used the same setup in my build process against the same IISExpress instance. This worked sucessfully.

3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - Request for https://localhost:44390/nuget - GET
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - No authorization header found
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - Request allowed by default
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Request for https://localhost:44390/nuget/ - GET
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - No authorization header found
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Request denied for /nuget/
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - Request for https://localhost:44390/nuget - GET
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - No authorization header found
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget - Request allowed by default
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Request for https://localhost:44390/nuget/ - GET
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - No authorization header found
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Request denied for /nuget/
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Request for https://localhost:44390/nuget/ - GET
3/05/2015 10:27:27 AM +00:00: https://localhost:44390/nuget/ - Password valid - setting the principal
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/ - Auth cookie set
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/ - User authenticated, request is allowed
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request for https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - GET
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - No authorization header found
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request denied for /nuget/
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Request for https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - GET
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Password valid - setting the principal
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - Auth cookie set
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/nuget/Packages(Id='OtherSystem.Client',Version='2.0.2') - User authenticated, request is allowed
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - Request for https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - GET
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - No authorization header found
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - Request denied for /api/
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - Request for https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - GET
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - Password valid - setting the principal
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - Auth cookie set
3/05/2015 10:27:28 AM +00:00: https://localhost:44390/api/v2/package/OtherSystem.client/2.0.2 - User authenticated, request is allowed

Why am I not seeing the same behavior when VSO build hits my custom NuGet server?

[yishaigalatzer]

How did you get nugetm.exe to vso? Is it exactly the same binary?

[roryprimrose]

I got it from the CommandLine NuGet package and put it in my repo. It is version 2.8.60318.667

[roryprimrose]

I also tested with nuget 2.8.50926.602 and found the same. With both these versions, I consistently get the problem locally. I don't know what I missed when I tested this locally the other day.

[bhuvak]

If you try downloading a package from your custom nuget server ( not locally hosted in IISExpress) via browser ( using /api/v2// ) does it work ? If there is a permissions related issue, it should fail.
Also when you try to hit your nuget.server from nuget.exe from your local machine, can you capture the fiddler trace and share it with us ?

[bhuvak]

@roryprimrose, I am closing the issue as we couldn't repro it and it seems like a permission issue.

[roryprimrose]

Sorry, I missed your last comment. I'll try to get a fiddler trace today.

I don't think it is a permissions issue. I have a workaround where I cache the IP address of a previously successful authentication attempt. I then allow that IP to make any call within a few minutes. This workaround (bad as it is for security) works fine both locally and hosted in Azure. Without the IP cache I get the same lack of authentication provision as described above.

[yishaigalatzer]

This looks a lot like a dup of #599

CC @feiling

[feiling]

@roryprimrose I checked our code and didn't find anything wrong. It does send out basic authentication headers.

Please use fiddler to capture all requests and responses. Then search for requests of "GET https://nuget.myserver.com/api/v2/package/othersystem.client/2.0.2".

There should be at least two requests. The response of the first request should look like this:

HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/8.5
WWW-Authenticate: Basic realm="xxx"
X-Powered-By: ASP.NET
Date: Wed, 20 May 2015 20:04:10 GMT
Connection: close
Content-Length: 0

Note the line WWW-Authenticate: Basic realm="xxx". This tells NuGet that the server needs basic authentication. So NuGet will send the same request, with this header

Authorization: Basic xxxx

where xxxx is the password.

Could you verify that the right response and request headers are there? If no, then it's a bug in NuGet and we'll fix this bug. If yes, then it's a setup issue with the server (most likely the username specified in your nuget.config does not have the permission to access the packages folder on the server).

[roryprimrose]

I spent a couple of hours working on this yesterday. I can't reliably reproduce this now. I looked at a restore scenario that uses both local and Azure based NuGet server. Looks like it is working on the command line ATM. Interestingly, a restore via the VS IDE fails though.

I still don't think it is a permissions issue because it works sometimes in some circumstances and works reliably with the IP cache. I just can't narrow down where this is going wrong. I'll close this for now unless I can reliably reproduce the issue.

[yishaigalatzer]

@nicholi One more question. Are you restoring/installing into a project with packages.config or project.json?

@martinsuchan 's follow up bug - #1158

[nicholi]

Still using packages.config. I haven't migrated to all the new nuget3/vs2k15 features just yet, but slowly moving towards them. I have migrated to the new automatic package restore method.

I have a feeling also just removing the sources and re-adding them with credentials again may have fixed more things then anything else. As I'm now ok even without @stevewilliamsuk's fix.

[martinsuchan]

@yishaigalatzer I'm not sure if that was an intended scenario for it or not, going to find out.
The "nuget.server" package is intended for running as ASP.NET app on IIS. There is no other scenario where it should be used, maybe on Mono or Azure, but that's basically the same.

I've even created simple tutorial how to deploy your NuGet Server on Azure and secure it with Basic Authentication. The repro for my bug mentioned earlier is based on this guide.

[johncmckim]

@nicholi if you lucky enough that re-adding it fixes it for you that's great. However, it definitely has not fixed the issue for our setup (can't tell you the number of times I tried removing and re-adding config). I have a very similar setup, Nuget.Server package and Basic Auth.

@yishaigalatzer do you have enough information based on all the examples and comments to reproduce the problem and look into it? If not what more do we need to provide?

It seems likely that bug is related to the difference between having https://somehost.com/nuget/ in the configuration and which isn't being picked up for requests going to https://somehost.com/api/v2/. Either it's a client issue, i.e. it should get the username / password per host or Nuget.Server should be using /api/v2/ instead of /nuget/.

Are you able to outline some kind of plan to resolve this issue? It's obviously a very frustrating problem for those people experiencing it.

As a follow up to the Nuget Gallery comment, if that's what we are supposed to be using, why is there no mention of it in the nuget documentation http://docs.nuget.org/create/hosting-your-own-nuget-feeds. I assume that many many people wanting to host their own feeds are following that guide and using the Nuget.Server package. I would be happy to use the Gallery project instead if it supports an Authentication system that I can hook into my LDAP server, but there's no mention of it in the docs as an option at all.

[yishaigalatzer]

@johncmckim I'm missing one bit of feedback.

The report from @martinsuchan talks about project.json scenarios. It is understood and we are tracking a fix.

I manage to connect to the feed and download the packages through VS2015 UI (though they are not applicable to NET452 projects they download fine).

So I still don't know if there is any other scenario broken here other than the one tracked at - #1158

@martinsuchan - Yes I wasn't aware it was (sorry relatively new to the team, I checked last night and yet it is supposed to work).

@johncmckim - Here is the difference between NuGet Gallery and nuget.server. Both support authentication, the former is more complicated to install, but provides a lot more features.

NuGet.Server
Works off the list of files on disk, becomes very inefficient to startup and find packages when the number of packages grows (above a few hundreds its going to start crawling)

Intended for small projects with a few packages. Mostly allows decoupling from using a fileshare, locking files and access over Http where fileshare access is not available.

NuGet Gallery

More of a mirror of the nuget.org V2 site, with a database behind the scenes and a lot more available features.

Here is one of the blogs I found on how to set it up with authentication. This might not be the most up to date one.
http://blogs.msmvps.com/p3net/2013/01/06/setting-up-a-private-nuget-repository/

You are right that the docs do not point to the nuget gallery solution, or go into the details above. I'll make sure that happens.

[nicholi]

No I was wrong, I started randomly getting prompted for passwords when manually installing/upgrading packages through the UI interface from the private repo. From what I've found you need to have multiple credentials entered for different paths, otherwise the client doesn't know what to do.

One for plain /nuget/ (I'm assuming this is for using the repo through the UI), one for /api/v2/ (for package restore to actually work), and I also have one for /api/v2/package/ (used when I push packages with api key). I have the last one disabled.

@johncmckim I'd make sure you have the latest nuget.exe and Nuget.Server package. Also try and assure the nuget.org repos are first in your list, and the protocolVersion=3 one is present. Otherwise yeah...fun and frustrating.

Ultra P.S. Also transition to the new automatic package restore method.

[jmyersmsft]

We're seeing some strange auth issues here too. Using nuget.exe 3.1.0-beta downloaded from the link in the blog post, with an authenticated v3 source with credentials stored in nuget.config, running

nuget list -source MyAuthenticatedSource

gives the following behavior:

1. GET (index.json) (without credentials) -> 401 with auth challenge
2. GET (index.json) (with Basic credentials) -> 200
3. GET (SearchQueryService URL) (without credentials) -> 401 with auth challenge
4. NuGet.exe gives up with the message "Failed to retrieve metadata from source: '[query URL]'.". No further HTTP requests are issued,

index.json and the SearchQueryService are on the same host, if that's interesting

[yishaigalatzer]

How did you get an authenticated v3 feed?

[emgarten]

We believe this is fixed in the latest version of nuget.exe. We have verified the following scenarios for basic auth:

• nuget.server hosted on IIS with basic auth
• authenticated v3 feed from myget.org
• project.json scenarios and packages.config scenarios
• proxy scenarios

I'm going to close this bug for now, use the link below to verify that this fixes your scenarios:
https://www.myget.org/F/nugetbuild/api/v2/package/NuGet.CommandLine/3.2-beta-10471

If you are still seeing issues with the above nuget.exe please open an issue with exact repro steps and a server that we can hit to test nuget.exe similar to how @martinsuchan set up the repro.

[martinsuchan]

This just happened on my machine - Windows 10, Visual Studio 2015 Pro with UWP Tools v1.1, NuGet 3.2.60914. When rebuilding UWP app sample -> about 100 Basic Authentication dialogs popped up even though I got our custom feed with saved basic authentication credentials properly added in feed sources. Note this sample used only default UWP NuGet package Microsoft.NETCore.UniversalWindowsPlatform, no package from our custom feed.
Bottom right shows the NuGet sources settings. This is a serious issue needs to be handled.
Note this issue is probably related to #1158

[yishaigalatzer]

Do you get a consistent repro? As in does this happen every time or just once?

We just did an overhaul for auth in 3.3 so it would be interesting to test it.

Last it doesn't matter if packages are in the authenticated source or not because the code cannot know that until they get queried

[martinsuchan]

It looks like I got consistent repro (Windows 10, Visual Studio 2015 Pro with UWP Tools v1.1, NuGet 3.2.60914):

1. download the repo of Windows 10 UWP samples
2. clean all already downloaded NuGet packages in C:\Users[user].nuget\packages to start with clean slate
3. make sure the custom NuGet feed with basic authentication has been added, see my post before
4. open this solution: Windows-universal-samples\Samples\CortanaVoiceCommand\cs\AdventureWorks.sln
5. start solution Rebuild -> lot of invalid Basic Authentication dialogs appear.

[martinsuchan]

I can confirm this is no longer happening with NuGet 3.3.0.

[yishaigalatzer]

@martinsuchan thanks for getting back to us! Much appreciated

[TKonov]

@stevewilliamsuk thank you for the insight and the suggestion. Works like a charm!