Cookies in 3.0

[tmenier]

Cookies never worked quite right in 2.x. Although you can get and set them using FlurlRequest.Cookies, that's deceiving because they actually live at the client level. This is due to a limitation with the mechanisms from the HttpClient stack. Flurl.Http 3.0 will get it right by turning off HttpClientHandler.UseCookies and processing the raw headers directly.

IFlurlRequest.Cookies and IFlurlResponse.Cookies

These read-only collections directly reflect the Cookie request header and Set-Cookie response headers for this specific request and response, respectively. Nothing fancy here; if the cookie wasn't sent or received in this specific request, it won't be in these collections.

Here's how you can set some per-request cookies manually:

var resp = await "https://cookies.com"
    .WithCookie("name", "value")
    .WithCookies(new { cookie1 = "foo", cookie2 = "bar" })
    .GetAsync();

These simply write to the Cookie header for this request only.

CookieJar

This is where things get a little smarter. CookieJar is a special collection type whose contents (FlurlCookies) contain the full details received in Set-Cookie response headers. When a CookieJar is passed to WithCookies, you get these additional behaviors:

• Values to be written to the Cookie header are filtered according to rules involving the origin URL, Domain and Path, expiration, etc, per RFC 6265.
• When the response is received, the CookieJar will be populated/updated with any Set-Cookie headers present in the response.
• The (updated) CookieJar is well suited to reuse in subsequent requests.

In short, CookieJar is Flurl's equivalent of CookieContainer from the HttpClient stack, with the notable advantage that it's not bound one-to-one with a client or handler.

You can create and populate a CookieJar (fluently of course):

var jar = new CookieJar()
    .AddOrUpdate("cookie1", "foo", "https://cookies.com") // you must specify the origin URL if populating manually
    .AddOrUpdate("cookie2", "bar", "https://cookies.com");

await "https://cookies.com/a".WithCookies(jar).GetAsync();
await "https://cookies.com/b".WithCookies(jar).GetAsync();

Or, if you don't need to pre-populate it, you can save yourself the trouble of creating it and use an out param instead:

await "https://cookies.com/login".WithCookies(out var jar).PostUrlEncodedAsync(credentials);
await "https://cookies.com/a".WithCookies(jar).GetAsync();
await "https://cookies.com/b".WithCookies(jar).GetAsync();

CookieSession

If passing that jar with every request feels like a drag, you can do it implicitly:

using (var session = new CookieSession("https://cookies.com")) {
    // set any initial cookies on session.Cookies
    await session.Request("a").GetAsync();
    await session.Request("b").GetAsync();
    // read cookies at any point using session.Cookies
}

That base URL passed in the constructor is optional; you can pass full URLs in the Request calls, or even a list of segments that will be Url.Combine'd together. These semantics are very similar to those of IFlurlClient.Request, and if fact, if you are using a FlurlClient explicitly, you can use new CookieSession(client) instead to make sure you're creating requests off that client.

Manipulating cookies

Typically, cookies are set/updated via Set-Cookie instructions sent by server, and if you use a CookieJar or CookieSession as prescribed above, things will just work. But if you do need to manipulate them directly, be aware of these behaviors:

• When you use WithCookies(jar), values relevant to this request are immediately written to the Cookies header. If you manipulate the jar after it is attached to the request and before the request is sent, those changes will not be synchronized with the Cookie header. So it's best to do any initializing/updating of the jar before attaching it to any requests.
• Successive calls to WithCookie(s) will overwrite values of the same name set in previous calls, regardless of whether they came from a jar or direct values. But other than overwriting by name, successive calls are additive.
• Once a cookie is added to CookieJar, it becomes immutable and you'll get an exception if you try to change it. Use jar.AddOrReplace to "update" a cookie.

Breaking changes

• Removed CookiesEnabled at all settings levels. It doesn't serve any purpose anymore. You're in direct control of sending cookies, you can't prevent a server from sending them in the response headers, and processing the Set-Cookies headers is done lazily, so there's no overhead involved when you don't need them.
• Removed Client-level cookies. This entailed removing the Cookies collection and WithCookie(s) methods from FlurlClient. CookieSession is the replacement, and it can even be managed as singleton via DI if that's appropriate. If there are legitimate reasons for client-level cookies I could be persuaded to add it back, but I can't think of any.
• FlurlRequest.Cookies is a now simple read-only list of Name/Value pairs. This more accurately reflects the Cookie request header, which doesn't contain all the details of the Set-Cookie response header. WithCookie(s) is the preferred way to write request cookies. (You could also manipulate the Cookie header directly, but I wouldn't recommended it in most cases.)
• Cookie and Set-Cookie headers are now visible in the Headers collections. This is a consequence of disabling HttpClientHandler.UseCookies, but I tend to like it better because it's a more accurate representation of the actual headers that were sent and received.

[Meberem]

I really like everything proposed here. Particularly the idea of out var cookies to get response cookies for a request and keeping the Cookie headers.

My current scenario involves a background hosted service (in asp core) to "login" and persist a Cookie and Controllers that make additional calls to the endpoint with that Cookie. Both places basically end up calling "endpoint.com".GetJsonAsync() so would need to share a Cookie Session. I'm wondering if a way to do that would be to have a Flurl.AspNetCore package that shared the Cookie Session in a similar way to Typed Http Clients.

[rcollette]

I'm curious why not address the underlying cookie issues in httpclient (whatever they may be) with the . Net core team?

What does my injection scenario look like for my DAOs. It kind of feels like I will need a service factory and then I make calls only using session? I'm tailing a bit off of @Meberem s question here.

Is session thread safe for updates. On a cookie token renewal will the cookie be locked at the appropriate time? There is a challenge here because as soon as the server sends a response and before parsing is complete, a prior token could be expired or invalidated.

[tmenier]

@rcollette

I'm curious why not address the underlying cookie issues in httpclient (whatever they may be) with the . Net core team?

dotnet/runtime#1904

I did my best but have heard nothing back so I'm not going to wait for them.

[tmenier]

@Meberem @rcollette An AspNetCore-specific companion lib is definitely something I'm considering, but I want to get 3.0 released first. Flurl.Http will continue to support a very broad range of platforms, and that's the context of this particular feature, so things like typed clients aren't really relevant to this. Let me know if you see any shortcomings, under the context that it is a platform-agnostic "building block" that may require a little work to marry it to an ASP.NET Core feature.

@rcollette It's possible that I'll need to switch to a thread-safe collection such as a ConcurrentDictionary for cookies. I hadn't thought of the possibility that the same "user session" could be hit by multiple threads, if that's what you're saying. I figured the more common scenario was different sessions on different threads concurrently (all using the same FlurlClient).

[tmenier]

This is now available in prerelease 4!

https://www.nuget.org/packages/Flurl.Http/3.0.0-pre4

There were enough changes since I first logged this issue that I rewrote the description almost completely. If you've been following this issue I'd encourage you to re-read it.

If you are using cookies with Flurl, testing this prerelease is absolutely crucial. It's completely different from 2.x, there's likely going to be some bugs, and it's extremely important to get the behavior right before the full 3.0 release, at which point it'll be too late for breaking changes. I greatly appreciate any and all feedback on this one!

[gpcaretti]

To be honest, this change disrupted one of the best features of Flurl. I use Flurl in a service invoked via IOC. Ver. 3.0, now requires me to handle explicitly cookies

[tmenier]

@gpcaretti If you post some details of what worked before and doesn't work now, I can probably help you with a smooth transition. Nothing with cookies should be harder than it was before.

[gpcaretti]

@gpcaretti If you post some details of what worked before and doesn't work now, I can probably help you with a smooth transition. Nothing with cookies should be harder than it was before.

Thank you very much.

My post intended that with v. 2.x it was enough to configure Flurl to handle cookies and all was in charge to it. This allowed me to wrap Flurl in a class library with a API such has:

public interface IWanAppService {
    Task Login(string username, string password, bool rememberMe);
    Task Logout();
    Task<ICollection<Court>> GetCourtsAllocation(DateTime date);
    Task<ICollection<CourtSlot>> GetSlots(Court cuort);
}

A simple API that I invoked via IOC in different ViewModel of my app:

    IWanAppService WanAppService => DependencyService.Get<IWanAppService>();

With v.3 my understanding is I have to create a CookieJar and handle it in my App, as it has to be passed to the business methods of my App because every invocation requires a WithCookies(jar) :

public interface IWanAppService {
    Task<CookieJar> Login(string username, string password, bool rememberMe);
    Task Logout();
    Task<ICollection<Court>> GetCourtsAllocation(DateTime date, CookieJar jar);
    Task<ICollection<CourtSlot>> GetSlots(Court cuort, CookieJar jar);
}

[tmenier]

@gpcaretti Hmm, ok, that's your service interface but it doesn't tell me much about how you're using Flurl. Are you injecting IFlurlClient? Registering it as a singleton? How are you using it in the service? (Request method?) And do you need just 1 cookie "jar" for the entire life of your application?

[gitfool]

FWIW, I had trouble deserializing CookieJar and FlurlCookie with System.Text.Json in .NET 5.0 and had to write another custom converter. See gitfool/BoardGameGeek.Dungeon@b0d04ff. (In the end I avoided working with CookieJar by using IEnumerable<FlurlCookie> instead and only dealt with the FlurlCookie ctor issues.)

[tmenier]

@gitfool feel free to log a new issue. It does make sense that persisting a CookieJar should be easy and if it's not maybe there's something that can be done about that. I'm open to suggestions on how.

[gpcaretti]

@gpcaretti Hmm, ok, that's your service interface but it doesn't tell me much about how you're using Flurl. Are you injecting IFlurlClient? Registering it as a singleton?

I had an initialize such has

    /// <summary>
    ///     Call once at startup
    /// </summary>
    public IWanAppService Initialize(HttpMessageHandler httpMessageHandler) {
        FlurlHttp.Configure(settings => {
            settings.CookiesEnabled = true;
            settings.HttpClientFactory = new MyHttpClientFactory(httpMessageHandler);
        });
        return this;
    }

Then all methods works in this way:

    public async Task<ICollection<CourtDto>> GetCourtsAllocation(DateTime date) {
        var json = new FlurlRequest(MYBaseUrl)
            .AppendPathSegment("/index.php")
            .SetQueryParam("param1", "myvalue")
            .GetJsonAsync();
         ...
    }

As you can see, the great thing I liked was that cookies where all "automagic", i.e. if a response set some cookies, the successive request automatically executed with the same cookies. All this thanks to the initial configuration.

With v.3 my understanding is I have to create a CookieJar and handle it in my App, as it has to be passed to the business methods of my App because every invocation requires a WithCookies(jar).