Baseline: what happens when a feature is removed?

[romainmenke]

example :

Firefox 113 will ship color-mix().
It is the last browser to do so.

One release later color-mix() will pass the current threshold for inclusion in Baseline.

But Safari's and Chrome's implementations are bugged.
At some point these bugs will becomes fully understood and MDN data should be updated to indicate this.

At that point in time color-mix() should be removed from Baseline because it no longer passes the requirements.

---

Is the label removed from the feature?

What if someone creates a tool to enforce "Baseline" in projects, what happens to developers who use such a tool?

[foolip]

But Safari's and Chrome's implementations are bugged.

Do you have links to bugs? We should add them as notes in BCD, and notes will be taken into account when distilling Baseline from BCD, that's the plan @ddbeck, @atopal and I have discussed at least.

Of course it's possible that serious bugs are found only after a feature is proclaimed to be Baseline. If that happens, then indeed we'll have to fix the data to say it's no longer Baseline.

If we think that doing this could be a breaking change to downstream tools using web-features, then we could consider bumping the major version of that release to signal a breaking change.

[romainmenke]

Do you have links to bugs?

I can't file bugs yet because the specification and WPT still need to be updated.
At this time we only know that what shipped isn't quite right.

This does however make it a great example for this scenario.
Sometimes it only becomes clear after shipping that not all cases were covered during the specification and implementation process.

If we think that doing this could be a breaking change to downstream tools using web-features, then we could consider bumping the major version of that release to signal a breaking change.

I think this is always a breaking change.

It's already nice that there will be semver on this data to communicate breaking changes.

[ddbeck]

For color-mix() specifically, is there an issue filed against the specification? Specification issues are an interesting risk—browser compat data rarely cites them because they're not implementation errors, by definition.

In general, I expect the risk of anticipated-but-unreported bugs is an area where subject matter experts are going to be key. If, for example, specification authors are made aware of Baseline, they can help us put the breaks on features that look fully-implemented, but aren't usable.

All that said, it's also possible that "Baseline with known bugs" is going to be a state that features get into from time to time. Sometimes notable bugs persist, but in a way that web developers can reasonably work around. We know we need a general way to include bug data alongside the Baseline status (e.g., to show why a feature is not Baseline), so solving that problem would help with this one too.

👍 to semver, though maybe we should revisit the idea of splitting the catalogue of features from the Baseline statuses, if consumers can't cope with status changes. It'd be nice to distinguish between "the set of features we've described has changed" from "the set of features in Baseline has changed" from "the definition of Baseline has changed" each of which could reasonably change independently of each other.

[romainmenke]

For color-mix() specifically, is there an issue filed against the specification?

Yes these have all been filed, but the issues themselves are not the point :)
This query surfaces some : https://github.com/w3c/csswg-drafts/issues?q=is%3Aissue+is%3Aopen+color-mix

browser compat data rarely cites them because they're not implementation errors, by definition.

Yes that is why I am not yet filing issues to adjust MDN data.
It first needs to become clear how implementations are going to change.

In general, I expect the risk of anticipated-but-unreported bugs is an area where subject matter experts are going to be key. If, for example, specification authors are made aware of Baseline, they can help us put the breaks on features that look fully-implemented, but aren't usable.

But this is a burden.

It requires specification authors and implementers to constantly keep track of which features might be included in baseline in the near future and then do extra chores to keep the baseline healthy.

All that said, it's also possible that "Baseline with known bugs" is going to be a state that features get into from time to time. Sometimes notable bugs persist, but in a way that web developers can reasonably work around. We know we need a general way to include bug data alongside the Baseline status (e.g., to show why a feature is not Baseline), so solving that problem would help with this one too.

This is true, but it is also very nuanced and requires a lot of context.

This context exists today in the compat tables, but it is erased when you add a green label at the top of a page stating that the feature is widely supported :)

[ddbeck]

It requires specification authors and implementers to constantly keep track of which features might be included in baseline in the near future and then do extra chores to keep the baseline healthy.

Sorry, I shouldn't have used the spec author as an example. Really I meant that people who really care about a particular feature will be instrumental. Sometimes they'll be spec authors or implementers, but web developers are subject matter experts too. I've got a lot of experience working on the data behind compat tables and often the best reports of issues or changes in support came from web developers themselves, who were interested in the feature but otherwise had no stake in the spec or implementation.

Apart from that, another area of work for Baseline is to make a distinct state for "forthcoming" or "coming soon" features. I expect that will be a really exciting addition for this kind of work in particular. Lots of possibilities once that's in place: increased attention from web developers, automation for the Baseline editorial process (e.g., a feature about to become Baseline gets a closer review), or even "intent-to-Baseline"-style notifications to implementers.

This is true, but it is also very nuanced and requires a lot of context.

This context exists today in the compat tables, but it is erased when you add a green label at the top of a page stating that the feature is widely supported :)

True, though there's some work to be done in BCD to make surfacing relevant notes and bugs more tractable (see mdn/browser-compat-data#17857 for example). BCD has lots of historical notes about bugs (I haven't analyzed this closely, but I suspect the overwhelming majority of bugs recorded in BCD were resolved a year or more in the past). Baseline might be a great way to help developers know when the compat tables need to be read carefully, or to call attention across fine-grained surfaces (e.g., if you're on the reference page for one method, but the overall feature-set feature group contains a serious bug in a related method, you'd never know that from a compat table alone).

[romainmenke]

To circle back to my questions :

Is the label removed from the feature?

It depends?

Maybe it needs to be removed, maybe context needs to be added in the Baseline widget?

Follow up :
If baseline is intended to be a simple and concise way to communicate that something is safe to use now, how do you add context to this without making it not simple, not concise?

How do you make sure that people understand all the nuance and the context and don't end up with a browser compat table at the top of each page?

What if someone creates a tool to enforce "Baseline" in projects, what happens to developers who use such a tool?

semver and changelog notes can be used to communicate data changes.

[atopal]

If baseline is intended to be a simple and concise way to communicate that something is safe to use now, how do you add context to this without making it not simple, not concise?
How do you make sure that people understand all the nuance and the context and don't end up with a browser compat table at the top of each page?

These things are hard to answer in the abstract. What we decided to do is to do some minimal design and decision making upfront, and then explore the feature space and to adapt our design as we encounter the need to add nuance. Our goal is to keep it a simplified feature status and to add nuance where that is helpful. We're already linking to the full compat table, and would rely on that in any case where nuance is clearly not possible in a concise way.

[romainmenke]

I think the current definition has enough escape hatches and opportunities for editorial choices to consider this issue resolved.

[ddbeck]

@romainmenke Thank you! I'll close this one now.