kestrel .net5 dateHeaderValues is null

[chris-kruining]

Hi,

I've been breaking my head on this issue for 2 days now, I followed the sample on IdentityModel.OidcClient.Samples to implement a local server to handle the oidc callback page. but I keep running into the issue of a null reference exception inside kestrel itself. Having delved deeper into this issue I have identified the source to be on aspnetcore/src/Servers/Kestrel/Core/src/Internal/Http/HttpProtocol.cs:1188. Looking into the call I have a suspicion it is "simply" a racing condition. since I have modified the sample a bit I'll post it below.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Security.Claims;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using IdentityModel.OidcClient;
using IdentityModel.OidcClient.Browser;
using IdentityModel.OidcClient.Results;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Serilog;
using Serilog.Sinks.SystemConsole.Themes;

namespace Fyn.Windows.Service
{
    public static class Authentication
    {
        private static readonly String Authority = "https://unifyned.cloud";
        private static readonly String Api = "https://unifyned.cloud/v1/cms/file";

        private static OidcClient? oidcClient;
        private static HttpClient apiClient = new HttpClient { BaseAddress = new Uri(Api), DefaultRequestVersion = new Version(2, 0) };

        public static async ValueTask Signin()
        {
            SystemBrowser browser = new SystemBrowser(5002);
            String redirectUri = $"http://127.0.0.1:{browser.Port}";

            OidcClientOptions options = new OidcClientOptions
            {
                Authority = Authority,
                ClientId = "Shell.Windows",
                RedirectUri = redirectUri,
                Scope = "openid profile email",
                FilterClaims = false,

                Browser = browser,
                IdentityTokenValidator = new JwtHandlerIdentityTokenValidator(),
                RefreshTokenInnerHttpHandler = new HttpClientHandler(),
            };

            options.LoggerFactory.AddSerilog(
                new LoggerConfiguration()
                    .MinimumLevel.Debug()
                    .Enrich.FromLogContext()
                    .WriteTo.Console(
                        outputTemplate: "[{Timestamp:HH:mm:ss} {Level}] {SourceContext}{NewLine}{Message}{NewLine}{Exception}{NewLine}", 
                        theme: AnsiConsoleTheme.Code
                    )
                    .CreateLogger()
            );

            oidcClient = new OidcClient(options);
            LoginResult? result = await oidcClient.LoginAsync();

            apiClient = new HttpClient(result.RefreshTokenHandler)
            {
                BaseAddress = new Uri(Api),
            };

            result.Show();

            await result.NextSteps();
        }

        private static void Show(this LoginResult result)
        {
            if (result.IsError)
            {
                Console.WriteLine($"\n\nError:\n{result.Error}");

                return;
            }

            Console.WriteLine("\n\nClaims:");

            foreach (Claim claim in result.User.Claims)
            {
                Console.WriteLine($"{claim.Type}: {claim.Value}");
            }

            Dictionary<String, JsonElement>? values = JsonSerializer.Deserialize<Dictionary<String, JsonElement>>(result.TokenResponse.Raw);

            Console.WriteLine("token response...");

            if (values == null)
            {
                return;
            }

            foreach ((String key, JsonElement value) in values)
            {
                Console.WriteLine($"{key}: {value}");
            }
        }

        private static async ValueTask NextSteps(this LoginResult result)
        {
            String currentAccessToken = result.AccessToken;
            String currentRefreshToken = result.RefreshToken;

            String menu = "  x...exit  c...call api   ";

            if (currentRefreshToken != null)
            {
                menu += "r...refresh token   ";
            }

            while (true)
            {
                Console.WriteLine("\n\n");

                Console.Write(menu);
                ConsoleKeyInfo key = Console.ReadKey();

                switch (key.Key)
                {
                    case ConsoleKey.X:
                    {
                        return;
                    }

                    case ConsoleKey.C:
                    {
                        await CallApi();
                        break;
                    }

                    case ConsoleKey.R:
                    {
                        RefreshTokenResult refreshResult = await oidcClient.RefreshTokenAsync(currentRefreshToken);
                        if (refreshResult.IsError)
                        {
                            Console.WriteLine($"Error: {refreshResult.Error}");
                        }
                        else
                        {
                            currentRefreshToken = refreshResult.RefreshToken;
                            currentAccessToken = refreshResult.AccessToken;

                            Console.WriteLine("\n\n");
                            Console.WriteLine($"access token:   {currentAccessToken}");
                            Console.WriteLine($"refresh token:  {currentRefreshToken ?? "none"}");
                        }

                        break;
                    }
                }
            }
        }

        private static async ValueTask CallApi()
        {
            HttpResponseMessage response = await apiClient.GetAsync("");

            if (response.IsSuccessStatusCode)
            {
                JsonDocument json = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
                Console.WriteLine("\n\n");
                Console.WriteLine(json.RootElement);
            }
            else
            {
                Console.WriteLine($"Error: {response.ReasonPhrase}");
            }
        }
    }

    public class SystemBrowser : IBrowser
    {
        public Int32 Port { get; }
        private readonly String _path;

        public SystemBrowser(Int32? port = null, String? path = null)
        {
            _path = path;
            Port = port ?? GetRandomUnusedPort();
        }

        private static Int32 GetRandomUnusedPort()
        {
            TcpListener listener = new TcpListener(IPAddress.Loopback, 0);
            listener.Start();

            Int32 port = ((IPEndPoint)listener.LocalEndpoint).Port;
            listener.Stop();

            return port;
        }

        public async Task<BrowserResult> InvokeAsync(BrowserOptions options, CancellationToken cancellationToken)
        {
            await using LoopbackHttpListener listener = new LoopbackHttpListener(Port, _path);
            await listener.Start();

            OpenBrowser(options.StartUrl);

            try
            {
                String? result = await listener.WaitForCallbackAsync();

                return String.IsNullOrWhiteSpace(result) 
                    ? new BrowserResult
                    {
                        ResultType = BrowserResultType.UnknownError, 
                        Error = "Empty response.",
                    } 
                    : new BrowserResult
                    {
                        ResultType = BrowserResultType.Success, 
                        Response = result,
                    };
            }
            catch (TaskCanceledException ex)
            {
                return new BrowserResult
                {
                    ResultType = BrowserResultType.Timeout, 
                    Error = ex.Message,
                };
            }
            catch (Exception ex)
            {
                return new BrowserResult
                {
                    ResultType = BrowserResultType.UnknownError, 
                    Error = ex.Message,
                };
            }
        }
        public static void OpenBrowser(String url)
        {
            // hack because of this: https://github.com/dotnet/corefx/issues/10361
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                url = url.Replace("&", "^&");
                Process.Start(new ProcessStartInfo("cmd", $"/c start {url}") { CreateNoWindow = true });
            }
            else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
            {
                Process.Start("xdg-open", url);
            }
            else if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
            {
                Process.Start("open", url);
            }
            else
            {
                Process.Start(url);
            }
        }
    }

    public class LoopbackHttpListener : IAsyncDisposable
    {
        const Int32 DefaultTimeout = 300_000;

        private readonly IWebHost _host;
        private readonly TaskCompletionSource<String> _source = new TaskCompletionSource<String>();

        public LoopbackHttpListener(Int32 port, String? path = null)
        {
            _host = new WebHostBuilder()
                .UseUrls($"http://127.0.0.1:{port}/{path?.TrimStart('/')}")
                .UseKestrel()
                .Configure(builder =>
                {
                    builder.Run(async context =>
                    {
                        switch (context.Request.Method)
                        {
                            case "GET":
                            {
                                await SetResult(context.Request.QueryString.Value, context);
                                break;
                            }

                            case "POST" when !context.Request.ContentType.Equals("application/x-www-form-urlencoded", StringComparison.OrdinalIgnoreCase):
                            {
                                context.Response.StatusCode = 415;
                                break;
                            }

                            case "POST":
                            {
                                using StreamReader sr = new StreamReader(context.Request.Body, Encoding.UTF8);
                                await SetResult(await sr.ReadToEndAsync(), context);
                                break;
                            }

                            default:
                            {
                                context.Response.StatusCode = 405;
                                break;
                            }
                        }
                    });
                })
                .ConfigureLogging(options =>
                {
                    options.AddSerilog(
                        new LoggerConfiguration()
                            .MinimumLevel.Debug()
                            .Enrich.FromLogContext()
                            .WriteTo.Console(
                                outputTemplate: "[{Timestamp:HH:mm:ss} {Level}] {SourceContext}{NewLine}{Message}{NewLine}{Exception}{NewLine}",
                                theme: AnsiConsoleTheme.Code
                            )
                            .CreateLogger()
                    );
                })
                .Build();
        }

        public Task Start()
        {
            return _host.StartAsync();
        }

        public async ValueTask DisposeAsync()
        {
            await Task.Delay(500);

            _host.Dispose();
        }

        private async ValueTask SetResult(String value, HttpContext context)
        {
            try
            {
                context.Response.StatusCode = 200;
                context.Response.ContentType = "text/html";
                await context.Response.WriteAsync("<h1>You can now return to the application.</h1>");
                await context.Response.Body.FlushAsync();

                _source.TrySetResult(value);
            }
            catch(Exception exception)
            {
                context.Response.StatusCode = 400;
                context.Response.ContentType = "text/html";
                await context.Response.WriteAsync("<h1>Invalid request.</h1>");

#if DEBUG
                await context.Response.WriteAsync($"<p>{exception.Message}</p>");
                await context.Response.WriteAsync($"<p>{exception.StackTrace}</p>");
#endif

                await context.Response.Body.FlushAsync();
            }
        }

        public async ValueTask<String> WaitForCallbackAsync(Int32 timeout = DefaultTimeout)
        {
            await Task.Delay(timeout);

            _source.TrySetCanceled();

            return await _source.Task;
        }
    }
}

It is the this line which triggers the exception

await context.Response.WriteAsync("<h1>You can now return to the application.</h1>");

Am I just an idiot and blind for a missing await somewhere?
Is the Heartbeat in kestrel internally broken?
Is my config of the WebHost correct?

Thank you in advance!

[davidfowl]

There were changes made to the heartbeat in 5.0 so it's definitely possible there's an issue.

What's the exception you're seeing?

[chris-kruining]

And it is dateHeaderValues that is null here

[davidfowl]

@chris-kruining do you have a reliable repro?

[davidfowl]

I wonder if this is related to using the WebHost directly vs using the generic host... My only guess is that somehow a request is being sent before the value is initially set. This shouldn't happen because we're supposed to do this during startup but clearly something else is happening here.

[davidfowl]

cc @halter73

[halter73]

@chris-kruining If you don't have a reliable repro which would be ideal, can you provide a stacktrace for where the NullReferenceException occurs? DateHeaderValueManager should be fully initialized by the time Kestrel's Heartbeat.Start() completes here:

aspnetcore/src/Servers/Kestrel/Core/src/Internal/KestrelServerImpl.cs

Line 160 in 045882a

ServiceContext.Heartbeat?.Start();

This is before Kestrel binds to any endpoints or processes any requests.

[chris-kruining]

at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.CreateResponseHeader(Boolean appCompleted) in /_/src/Kestrel.Core/Internal/Http/HttpProtocol.cs:line 1193

I'm afraid a single line stacktrace wont help much :S

I'll try to set up a repro repo instead.

[chris-kruining]

https://github.com/chris-kruining/dotnet5-kestrel-issue-repro here you go

I haven't removed all logic around it in the case it's my code that's wrong.

[Kahbazi]

@halter73 I was a little curious and checked the code. Here's what I found.

There's an exception in OnHeartbeat method.

System.TypeLoadException: Could not load type 'Microsoft.Extensions.Primitives.InplaceStringBuilder' from assembly 'Microsoft.Extensions.Primitives, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.
at Microsoft.Net.Http.Headers.DateTimeFormatter.ToRfc1123String(DateTimeOffset dateTime, Boolean quoted)
at Microsoft.Net.Http.Headers.HeaderUtilities.FormatDate(DateTimeOffset dateTime, Boolean quoted)
at Microsoft.Net.Http.Headers.HeaderUtilities.FormatDate(DateTimeOffset dateTime)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.DateHeaderValueManager.SetDateValues(DateTimeOffset value)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.DateHeaderValueManager.OnHeartbeat(DateTimeOffset now)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.Heartbeat.OnHeartbeat()

@chris-kruining This project is using kestrel nuget package with version 2.2 which depends on InplaceStringBuilder and it's removed in 5.0.
Based on what I run. Removing Microsoft.AspNetCore.Server.Kestrel package and setting the project sdk to Microsoft.NET.Sdk.Web should fix the problem.

[chris-kruining]

@Kahbazi Can I swap that sdk??? I am making a wpf desktop app, just need a localhost listener to handle the callback of oidc

[Kahbazi]

As far as I know it's not a problem. Using the web sdk adds the packages you need for listener which is Kestrel, but just to be sure you could wait for @davidfowl or @halter73 to give the final answer.

[davidfowl]

@chris-kruining you don't need to change the SDK. You can just add a framework reference to ASP.NET COre:

<ItemGroup>
+    <FrameworkReference Include="Microsoft.AspNetCore.App" />
-    <PackageReference Include="Microsoft.AspNetCore.Server.Kestrel" Version="2.2.0" />
</ItemGroup>

[davidfowl]

Thanks for looking @Kahbazi !

[chris-kruining]

works like a charm!

For my understanding, what is it I did wrong? because 2.2.0 is the latest version on nuget. And I feel like adding a whole framework for a single feature is a bit overkill. Is FrameworkReference the norm over PackageReference. Or is this an exception because the nuget package is out of date?

[davidfowl]

I'd recommend reading this https://docs.microsoft.com/en-us/aspnet/core/migration/22-to-30?view=aspnetcore-5.0&tabs=visual-studio#framework-reference

[ilharp]

Wooooooooooooo

I've been breaking my head on this issue for 2 days now

exactly the same

---

One difference is that I try to run multiple Hosts while using PackageReference in a single Console app. The problem disappears after modified to FrameworkReference. (Commit)

  <ItemGroup>
+    <FrameworkReference Include="Microsoft.AspNetCore.App" />
-    <PackageReference Include="Microsoft.AspNetCore" Version="2.2.0" />
-    <PackageReference Include="Microsoft.AspNetCore.Http.Connections" Version="1.1.0" />
-    <PackageReference Include="Microsoft.AspNetCore.Mvc.Core" Version="2.2.5" />
  </ItemGroup>