Proposal: Allow servers to take full responsibility for cross-origin access protection #878
Comments
|
Is this a serious proposal for something new, or mostly just a complaint about CORS changing? (I'm not saying the complaint isn't valid) |
|
This is a very serious proposal. We are working with public APIs, and recently saw applications break. We want a sustainable mechanism for the server to take control of cross-origin protection. (For insight into our serious intentions, see the discussions in #865 and #862; the above proposal is the result of several weeks of drafting and rewriting). |
|
But the reason you don't see CORS as the solution is (1) multiple headers needed and (2) recent changes to CORS, or am I missing something? |
|
As described in Shortcomings of current mechanisms above, the problem is 1) the complexity (which is broader than the need for multiple headers; also the fact some of those headers need to be reactive, which is harder to configure in NGINX/Apache), but mainly 2) that no such configuration is a final solution. We understand that the fetch spec is in evolution, as it should be, but this currently means that all public APIs and authenticated APIs have to keep on updating to be able to function like before. And since the solution is on the server side, Web apps have no control over breakage. |
|
I don't think any solution can be final. If you invent yet another opt-in, say The CORS change seems bad, but I don't see yet another opt-in making things easier. Another opt-in has the same server-updating problem. |
That would make stable public APIs and authenticated APIs an impossibility. I think we can do better.
The trick is in letting server operators understand exactly what they are opting in to. I challenge anyone to ask 10 server operators what they are opting in to when they are providing So this is not about opting in to certain features. It is a matter of saying
So let's not have another opt-in then, but rather something sustainable. |
I don't think that's true.
What's your proposal, if not an opt-in? |
It is: dozens of Web APIs are running that have followed https://enable-cors.org/ with the intention of working from any Web application. These instructions no longer hold, so those APIs need an update on the HTTP level (= not stable). Furthermore, any such changes can happen in the future, so APIs will have to make adjustments (= not stable). Hence, I conclude that stable public APIs are not a possibility if servers do not have the ability to explicitly take responsibility for cross-origin access protection.
A mechanism by which a server can say "I am taking care of all current and future cross-origin protections for this resource", and/or "this is a non-personalized resource". So not a matter of opting in or out from specific features (which CORS is designed for, and still useful for), but rather deciding who assumes that responsibility. As such, servers accessed from the browser would have the same protections as when accessed from the command line or a native application. So apps can do nothing from the browser which they wouldn't be able to do through other means anyway. |
I don't know the history behind the CORS change. It seems pretty bad that a large breaking change was made. But, if that change was justified, similar future issues would be a problem for your proposal too.
You're describing an opt-in. |
My proposal/request is exactly to be immune from such issues. Probably some security bug exists where long headers cause problem X or Y (@annevk couldn't publicly comment); hence the change. I'm describing a mechanism where the server says: you know what, if X or Y indeed are problems, then nothing additional is compromised compared to doing the same attack from the command line or a native app. Hence, I don't rely on the browser security mechanism. (This is very different from cases where, let's say, an API does cookie-based authentication, in which browser-based requests would have a privilege over other means.)
Given that any mechanism could be characterized as opt-in (e.g., when I'm Accepting text/json, I'm opting in to JSON), I'll need better definitions to meaningfully address your concern with an opt-in. I do not share the view that opt-ins will necessarily need adjustments, if a server knows exactly what it is opting in or out of, and if that is broad enough to cover the above cases. |
This guarantee cannot be made. If a vulnerability is discovered that puts real users at risk, browsers will fix that to protect users. If the spec doesn't update, then the spec won't reflect reality. If a browser doesn't update, users and competitors will rightly accuse that browser of being less secure than other browsers. |
That isn't true. Only opt-ins can be characterised as opt-in. For instance, if you added this new behaviour by default, with a way for a server to say "I don't want this", then your proposal isn't opt-in (and an opt-out is provided). |
|
If a vulnerability is discovered that puts real users at risk, browsers will fix that to protect users.
The server needs to update. Just like browsers are not responsible for
server-side input sanitation against SQL injection, they are not
responsible for servers that explicitly take responsibility of cross-origin
request protection.
|
Indeed, what I am proposing is an opt-out. An opt-out of browser-side cross-origin protections (because the server takes care). I did not call it an opt-in. |
|
In/out doesn't matter. It's the 'opt' that's important. It's something the server has to opt for. |
I find it hard to argue this way. I am told that opt-in doesn’t work, that opt-out is not opt-in, I explain that it is opt-out, but then am being told the difference doesn’t matter. So yes, the server opts for taking responsibility of cross-resource protection, just like it already has other responsibilities. What is the issue with that (given that servers are already trying to en masse)? |
|
If the browser adds a new network capability, the responsibility of the server has changed without them being notified. If this results in a site's users being compromised, it's the browser's fault, because it was their change that broke things for users. This is why we wouldn't introduce the new capability without an opt-in. On one hand you say:
Then on the other:
|
Where did I say "opt-out is not opt-in"? |
|
Today servers express a clear wish to opt-out through Instead of having this walk-around servers now implement, we propose that a dedicated mechanism is put in place. This mechanism then becomes part of the living specification, where browser-vendors would be more cautious (never say never) to change anything on this part as it may break these applications. |
|
Let me try and explain: If I said to you "Here is a button, if you press it, it gives me £20 of your money", nothing has changed by default other than the existence of the button. If you press it, I get £20 of your money, but it's fair to say you have opted in to this transaction. If I just took £20 of your money, this was not an opt-in experience for you, because you were not consulted ahead of the transaction. It wasn't optional. If I was going to take £20 of your money unless you pressed a button to prevent me, default behavior has still changed. You haven't opted in to giving me your money. You could say there's an opt-in feature to prevent me getting your money, but because the button reverts things to default, it's more commonly called an opt-out. Similarly, you could frame CORS and your proposal as opt-ins or opt-outs, but they're definitely one of those, and they're both the same one. If, for serious security reasons, breaking changes were made CORS, those same reasons would apply to your basically-the-same proposal. If a significant new capability arrived that required an opt-in, users of your proposal wouldn't bypass that opt-in unless it could be proven to be safe. This is basically how CORS started. A new capability (cross-origin XHR) was introduced that was unsafe to enable by default, so an opt-in (CORS) was created. |
You're describing CORS. |
We are indeed proposing additions to the fetch specification related to CORS to tackle the use cases that are not tackled today. See |
|
I see the list of shortcomings, but nothing to show how the new thing avoids the same pitfalls (aside from the things already covered by https://github.com/WICG/origin-policy). |
That seems like a reasonable interpretation of
First of all, we are following the recommended the procedure at https://whatwg.org/faq#adding-new-features and thus proposing a problem that should be solved, not a solution. So "our proposal" (= a problem to be addressed) cannot be equal to "CORS" (= a solution). What we are saying is that the above use cases are not addressed by CORS, and we have received no indications of the contrary. CORS is a method for relaxing very specific conditions of the cross-origin protection mechanism. CORS does not provide a method to complete opt out of cross-origin protection (if it does, please let us know). At this stage, it seems highly preliminary to discuss potential solutions and their drawbacks, or to argue for the non-existence of any solution. We are currently looking for arguments that prove or disprove the validity, relevance, and importance of our use cases. Only when we have agreed that they are a problem that is not addressed currently, we should look into the possibility of creating and discussing solutions. |
|
In terms of opt-out/in, I can only point back to the examples in #878 (comment). It still sounds like a summary of the OP is "CORS behaviour shouldn't have changed". |
That's not what I wrote at all; please don't misrepresent my use cases. CORS can be changed in a thousand ways. I just don't want servers that explicitly do not want any cross-origin protections whatsoever, in whatever shape of form, to be affected by any such changes. Servers that do not explicitly opt out of such protections, or who only granularly opt in to CORS using the existing mechanisms, will and should be affected by all the changes. So if anything, I'm advocating for a "do not bother me with CORS protection" switch, precisely because CORS will and needs to keep on changing in the future. Very different from "CORS behaviour shouldn't have changed". Update: although I do not see how the original text could be interpreted as "you shouldn't have changed CORS", I nonetheless added an explicit section which explains that CORS changes are good and necessary. So let it be clear that we are advocates for secure CORS. |
|
@jakearchibald I think it's fair to say that there are two points - one which is a suggestion that existing functionality should not have changed, and another suggesting there should be a guarantee not only that existing functionality won't change, but that servers won't need to change to take advantage of new platform features (or relax any newly tightened restrictions on existing features) I think both could have some parallels drawn to the discussion between Mixed Content. Historically, UAs were very lax in their permissiveness of both active and passive mixed content. Developers were allowed to choose what they felt was the appropriate trade-off between security/privacy and functionality, and thus would very often load HTTP content into the context of HTTPS, exchange cookies around between the two, etc. The desire is reasonable, but I think it runs into challenges when faced with the Priority of Constituencies and Secure by Design The former prevents being able to guarantee that nothing will ever change - as UAs need to ensure that the User's needs and wishes are first and foremost respected - while the latter means that new features need to consciously consider whether there are risks to introducing them, and if so, ensure that they are introduced in a way that can safely reasoned about. The problem with a default opt-out is that it cannot be safely reasoned about, because at the time the opt-out was made, the information wasn't available. There's no way the developer could have made a (truly) informed choice, and it seems that some of the discussion of the problem is really a difference in philosophy about whether or not the developer was making an informed choice. The problem with guaranteeing there won't be behaviour changes is that, as you highlighted, our understanding of the Web Platform and its security evolves over time, as do the needs of the users, and so that the Web evolves around those. This is fundamentally reflected in the nature of this spec being a Living Standard - things change. Independent of exploring solutions, it may be that there is a fundamental disagreement on the nature and validity of the problem, and whether or not web developers or user agents should be the arbiters of user security, both presently and in the future. This is an inherent tension UAs face - users want and benefit from powerful new functionality and features that enable otherwise inaccessible use cases, but users also want privacy and security and safety when interacting with the Web Platform. @RubenVerborgh Do you feel that I've accurately captured some of the tension in perspective? |
Unfortunately not.
I did not make that point and I do not agree with that point. It is good that CORS has changed. It protects those servers that only wanted to selectively disable some cross-origin protections, which is what the CORS headers provide.
I did not make that point and I do not agree with that point. I think CORS functionality should keep on changing to protect those servers that only want to selectively disable some cross-origin protections.
I am not arguing for a default.
Why not? If I know that my resources are public and not personalized, what is not informed about my choice to say, I will take indefinite responsibility for cross-origin requests for these specific resources? |
|
@RubenVerborgh Apologies for misunderstanding your point, then. To make sure I'm accurately understanding your point and accurately presenting it:
Is that better? |
|
Thanks for trying to summarize @sleevi, that's helpful.
Fair enough. If a server says, "I don't want feature X", then it is indeed also opting out of all future alterations to X. Just like, as a server, when I don't provide content negotiation, I am also opting out of content negotiation changes in the future. That is indeed very much the point.
I will remove the mentions out of enable-cors from point 8, they can indeed be confusing. |
|
A number of your use cases (2, 4, and 7) refer to authentication or access control. I think understanding whether those use cases are being addressed securely (and, e.g., not in a way that's subject to the confused deputy problem) requires understanding what the use cases for that authentication or access control are. That is, why is authentication or access control being used, and is this solution sufficient for that reason? Also, one other side comment: while |
Thanks @dbaron, that's a very relevant point indeed. So, broadly (and very loosely) speaking, I see four categories of authentication between browser apps and a server:
Clearly, cross-origin protection is necessary for the second category, or the danger is that a web app reads privileged information. It is also necessary for the third category, if the client uses The problem is that the browser doesn't know in which of the categories it is, so it makes a pessimistic assumption (for instance, that it is in category 2). That makes sense, given no explicit indication of which category it is in. However, cross-origin protection is also applied in categories 1 and 4, and undesirably so, if the browser is told by the server in which category it is. In category 1, the resource is public. So anyone from anywhere in the world sees the same thing; as such, the user's information cannot be compromised. So if the server were to state "this is a public resource", then not having cross-origin protection (now or in the future) is perfectly acceptable. In category 4, evil.com cannot access the resource unless the user has authorized evil.com, in which case an evil.com-specific key will be sent to the server. So again, no need for browser-side cross-origin protection here if the server indicates that it is taking care of cross-origin protections, because another mechanism is active that handles these protections. We do not have a confused deputy problem in these categories. In 1, anyone can access. In 4, access for the specific origin is regulated through another mechanism.
Given that, as proposed, the server explicitly indicates that it takes control of cross-origin protections, then: In category 1, no authentication is used/needed at all (so sufficient).
Well, yes and no. Yes, in that several cases work; no, in that it has become difficult to provide an exhaustive list of conditions that requests have to satisfy before being usable—or, conversely, an algorithm to generate the necessary headers for any request to be usable. And any such a list or algorithm would not be stable. Understood that such changes happen for security reasons, but the above categories 1 (no auth) and 4 (separate auth) are not impacted by them if the server chooses to regulate cross-origin protection itself. So that's why I am arguing for a sustainable solution for those categories. (Note: categories 1 and 4 correspond to scenarios 1 and 2 in the section “Problem description”.) |
|
I could see a TLS-level assertion that the server is basically open to all kinds of connections/requests and is able to protect itself as a thing that might be workable: quicwg/base-drafts#1993. There's a lot of risks for the server involved though as I outlined in a comment there. For CORS itself it seems worthwhile to wait on how Origin Policy/Manifest works out. |
That is a very interesting direction indeed; opens up some possibilities.
Origin Policy is definitely another interesting direction. However, currently, the |
annevk
added
topic: same-origin policy
addition/proposal
|
Another frequently used public API affected: schemaorg/schemaorg#2412 |
|
One clear and reliable method to fix this is: Make
|
|
From my perspective, this problem is best stated in terms of invariants: What does WHATWG guarantee will hold in all future versions of the fetch specification, and how can a server signal that they only rely on these guaranteed properties? One candidate for an invariant is No origin can access or tamper with the body of a request sent by a different origin without authorization by the sending origin or the user agent. If this invariant holds, then any server that ignores request headers will not be affected by future changes to the fetch specification. |
TL;DR: Servers that explicitly take full control of cross-origin access protection, do not want the browser to handle this. Unfortunately, fully and indefinitely opting out is currently impossible.
Following the WHATWG procedure for feature proposals, this issue describes the problem and use case requirements, not a solution (yet).
Problem description
By default, browsers are responsible for protecting cross-origin access to resources. This mechanism was created to avoid scripts running on one origin from having undesired access to personalized content on on another origin.
Currently, there is no sustainable way for a server to take full responsibility of cross-origin access protection. Servers can selectively opt out of blocking behavior, but no opt-out mechanism is guaranteed to work for all current and future applications. As an example, we recently witnessed breakage of several legitimate applications that relied on a widely used server-side configuration from enable-cors.org, because the fetch specification had changed in subtle ways. The only resort is reconfiguring all servers, without guarantee that this will be a permanent solution.
While useful as a default, the browser’s exclusive and changing control of cross-origin access creates an undesired obstacle in two common scenarios:
when the server does not provide any personalization for a resource (“open data” or “public APIs”);
when a resource’s personalized behavior is secured through other means such as API keys or authentication headers sent by the requesting script (“authenticated APIs”).
Use cases
(1) A Web server does not provide any personalization of certain resources, and wants to make those available to any Web application, now and forever.
Requirements:
(2) A Web server has its own cross-origin authorization mechanism for certain resources, and wants to make those available to any Web application, now and forever.
Requirements:
(3) A Web application that makes cross-origin requests to public resources on a certain server wants to keep working, now and forever (given no changes on the server).
Requirements:
(4) A Web application that makes authenticated cross-origin requests to resources on a certain server wants to keep working, now and forever (given no changes on the server).
Requirements:
(5) A browser wants to move the responsibility for granting access to cross-origin resources to the server, when requested.
Requirements:
(6) A browser wants to maintain the possibility of providing granular cross-origin access protection to servers that do not explicitly opt out of this protection
Requirements:
(7) A server developer wants a dedicated mechanism for taking server-side responsibility for cross-origin access control
Requirements:
(8) A developer website (such as developer.mozilla.org) wants to document a future-proof way of taking server-side responsibility for cross-origin access protection.
Requirements:
Shortcomings of current mechanisms
Currently, when trying to address the above use cases, servers must resort to multiple HTTP header settings that eliminate cross-origin request blocking by enabling Cross-Origin Resource Sharing (CORS). In contrast, their actual goal is to request full responsibility for this protection. Therefore, the fact that only fine-grained settings are available is problematic, because:
It is complex to indicate that a server requests responsibility for all current cross-origin requests, as this configuration involves an interplay of several HTTP headers with subtle edge cases.
It is impossible to indicate that a server requests responsibility for all future cross-origin requests, because of continuing changes to the fetch specification that tighten the mechanism. As such, legitimate Web applications relying on CORS can break at any time.
Clearly, this complexity and progressive tightening are beneficial for the protection of the user and servers in general. However, this proposal argues that there are many common cases where the server explicitly wants to take that protection in its own hands: public data, open APIs, authenticated APIs.
Recent changes in the fetch specification broke widely deployed instructions on how to disable CORS. For instance, Web applications using HTTP requests with long headers suddenly stopped working in 2018/2019 after browser updates, even though their servers followed configuration instructions with the explicit intention of this not happening. Fixing those applications requires changes on the server side. Getting all affected Web servers updated is expensive and will likely take several years, and there is no guarantee that such an update will not be obsoleted again. It is unsure whether troubled servers will be updated in timely manner or at all, because blocked requests do not show up in server logs, so servers have no way of knowing that applications have trouble accessing their resources.
Note that we are not arguing against past or future changes to CORS. For security reasons, it is beneficial and necessary that the fetch spec keeps on updating. Rather, we are arguing for the existence of cases in which the server a) does not need that security because it it an open API, or b) is already taking the burden of security by authenticating cross-API requests in different ways.
We thus argue that a considerable number of servers emitting the
Access-Control-Allow-Origin: *header actually aim to express their wish to take control of cross-origin request protection themselves, and thus for the browser to fully delegate that responsibility, instead of the much more nuanced and limited meaning this header actually has. These servers thus need a proper way of expressing this, without having to rely on the misinterpretation of an existing header.Current Web applications
The following Web applications seemingly have the intention of requesting full server-side control of cross-origin access protection. Instead, they resort to workarounds which, as argued above, can break at any point (and, in multiple cases, are currently broken):
curl 'https://data.opendatasoft.com/api/records/1.0/search/?dataset=gare-de-train-et-tramway%40sarthe&facet=nom&facet=ligne&facet=typ_gare' -IAuthors of this proposal: Ruben Verborgh (@RubenVerborgh) and Pieter Colpaert (@pietercolpaert).
The text was updated successfully, but these errors were encountered: