Define how the core browser set was determined

[tomayac]

Based on an interesting question asked on Mastodon by @dgp1130. I was intrigued, but couldn't find based on what criteria the core browser set was determined.

[ddbeck]

Hi @tomayac, I'll do my best to answer your as succinctly as I can. I'll respond to the more specific "What if…?" question separately.

In the strictest terms, the core browser set was selected by the web-features owners group in a consensus-seeking decision-making process. We worked on the overall definition, including the browser set and releases timeline, until we reached an actual consensus. This process took about 6 months. Most discussions happened in WebDX meetings, the WebDX matrix chat, and in issues labeled here.

More broadly, we picked the browser set to accommodate a number of requirements and constraints:

• Developer expectations (e.g., through surveys, direct feedback on an earlier definition, and representation in the owners group)
• Developer behavior (e.g., looking at, for instance, open source browserslist configs)
• Baseline consumer expectations (i.e., what would work for caniuse and MDN)
• Upstream data availability and quality (i.e., what data sources were actually available to us, for support and usage)

A lot of work went into trying to find some statistical rule (e.g., using StatCounter usage thresholds) that would encompass everything. Unfortunately, data quality and other issues prevented that from happening. So the governance group reviewed all the information and made a judgment about a browser set that we hoped would be satisfactory to developers.

The formal Baseline definition document contains a lot of information about our shared motivations, at the time it was adopted.

[ddbeck]

To @dgp1130's question, which I'll excerpt here:

Presumably the browser set would evolve if a new browser gets enough market share to justify its inclusion.

What happens if that browser is missing a particular feature supported by the other browsers?

Does that imply that Baseline 2025 could hypothetically include less features than Baseline 2024 if it included a new browser? I feel like losing features is something you wouldn't expect in Baseline, much like you don't expect a version to go backwards.

Is there any expected outcome for this particular scenario?

When we adopted the Baseline definition, we did not resolve to answer the question of what the meaning of Baseline would be before 29 July 2015 (the first release of Edge) or after some future change to the core browser set. We talked about it and speculated on it but, if I remember correctly, the consensus was that it would be premature to work out a definition versioning scheme or transition policy before we had demonstrated that Baseline was useful to developers.

We did commit to an annual review of the definition, to affirmatively decide whether it was still working or to kick off a revision process. Since being unsurprising is part of Baseline's goals, I can't imagine we'd make a change to the core browser set without a lot of advance consultation and notice.

I want to note one last thing, though:

I feel like losing features is something you wouldn't expect in Baseline, much like you don't expect a version to go backwards.

It is possible for a feature to move backwards, even with the existing browser set, though it should be a rare event. For example, suppose a feature reaches Baseline widely available status but a browser subsequently unships feature (or ships a bug so serious that it's close to the same thing). We'd have to "un-Baseline" that feature and the feature would have to progress from limited availability through newly available again. Thankfully all of the examples I have for this, such as IndexedDB, are increasingly historic—but there's no rule that says it can't happen again.

[dgp1130]

Thank you for this context @ddbeck and thank you for following up on my question @tomayac!

What you describe around losing features does sound reasonable. SharedArrayBuffer also comes to mind as an instance where a feature was unshipped cross browser, though like you mention that's a pretty rare event on the web which isn't too much of a problem in practice. It does make sense that if the browser platform moves backwards by unshipping or breaking a feature, then Baseline does need to align with that, even if it might be counter-intuitive.

So if I'm understanding correctly, it sounds like exactly how a new browser would be introduced into Baseline is still somewhat ambiguous and undefined and will likely stay that way until such a scenario actually happens? That sounds reasonable given the context and the way these kinds of decisions tend to get made, though I just want to point out that this does have an ecosystem impact.

Any library or tool which commits to Baseline support ("We will only require browser APIs which are Baseline as of any given version's release date") doesn't really know what it's committing to in the situation of a new browser addition. As you mentioned, Baseline can already "go backwards", but those cases are relatively exceptional. New browsers are somewhat common, but reaching the threshold for inclusion in Baseline is likely just as, if not more exceptional than unshipping / breaking existing features. So I don't think this is a major problem, but it does mean that any ecosystem tool adopting Baseline as a browser support policy needs to consider what happens in the scenario that a new browser is added. With that currently being undefined in Baseline itself, it's hard to know exactly how to handle that case. The most likely option for a versionable tool would be something like "We will only require browser APIs which are Baseline as of any given version's release date, with the exception of versions X1-X2 which do not support browser Y."

All that said, a new browser would certainly be an exceptional circumstance and any versioned tool can adjust its browser policy to match since Baseline support is tied to a specific date, so adding a new browser would presumably not retroactively change Baseline for previous dates. As a result, I don't think this is a major problem, but something to be aware of. Maybe it would make sense to document somewhere that this scenario is still undefined? Or maybe at least commit to not changing the definition of Baseline support for dates prior to inclusion of the new browser (that part feels uncontroversial to me and would help at least versioned tools to adopt Baseline). But otherwise I can imagine that we likely won't see consensus on actually defining it until this scenario really happens.

Thanks for the discussion here, this is a really interesting edge case and I appreciate the added context!

[ddbeck]

Briefly, summarizing my to do list here:

• Document undefined-ness of Baseline in the eras before and after the present core browser set (Baseline defintion: flag undefined status situations #2142)
• Document guarantees (if any) in the case of adding a new browser to the core browser set

---

Maybe it would make sense to document somewhere that this scenario is still undefined?

Absolutely. I've opened #2142 to (begin to) cover this.

Or maybe at least commit to not changing the definition of Baseline support for dates prior to inclusion of the new browser (that part feels uncontroversial to me and would help at least versioned tools to adopt Baseline).

Yes, I think it's a good idea to make some sort of guarantee about this, even if it's minimal (e.g., some sort of notice period before making changes to the core browser set). As for the specific shape of that guarantee:

Do I understand correctly that you're suggesting that we commit to freezing the browser set for a feature at the time it reaches Baseline status? That is, once a feature reaches Baseline, we'd only score that feature on the (ongoing) support from the core browser set at the time?

If I have that right, I'm not sure if we can make that strong a commitment. Over the long haul, I'd expect that if developers cared enough about the new browser to warrant its inclusion that they'd also eventually stop relying on features that don't work in that browser, no matter the historic case for marking that feature as Baseline. But that's an assumption and one I'd like to test a bit. I think there are other plausible routes (e.g., a sunset period or an explicit "legacy" or "vested" status for features that have this quality) and I'd like to do more intentional research into developer expectations about all the options before committing to one.

An easier commitment today might be be something like: introducing a new browser won't cancel a Baseline status for an existing feature for at least 30 months (e.g., an off ramp that mirrors the widely available time horizon). Telegraphing the removal of a feature from Baseline—if we know it's going to happen in advance—seems really valuable.

This is somewhat related to #311 and #955, where we need the ability to communicate a probable future status (i.e., shipping or unshipping anticipated in a future release). We'll actually have to resolve that sooner than any change to the core browser set, since feature deprecation or removal is a rare but more frequent than the appearance of a new must-support browser.

This is also related to #1971, where we should explain (as part of the data) why a given feature has regressed, if it does.

[tomayac]

(Just wanted to thank @ddbeck and @dgp1130 for the discussion. I'm an interested silent observer [and the messenger in this case], so don't block on a response from me.)

[dgp1130]

@ddbeck, I think you might be interpreting this support question in the context of supported features, but I'm actually thinking about this from the set of supported browsers. I'm looking through the lens of "What are the Baseline browser versions?". This is a slightly different question from "What are the Baseline features?", which is a separate artifact that is derived from the set of browser versions. For libraries and frameworks, the goal is partially to identify what features are available in Baselines, but also to identify what workarounds might still be necessary to support Baseline browsers. So in this context, I think I care less about feature support and more about the precise set of versioned browsers at a particular time.

Let's look at this from the context of a strongly versioned library/framework released on a particular date. My thinking is that such a tool might use the "widely available" Baseline by defining its browser support as:

Version ${V} of this library/framework/tool supports the "widely available" Baseline as of date ${D}.

Date ${D} is likely the release date of that version of the tool, or close to it. In practice, this means that the "widely available" Baseline would take ${D - 30 months} and get the then-latest versions of the core browser set (call this ${B}). The minimum set of features supported by those browsers provides the Baseline feature set (call this ${F}).

The above support policy essentially states:

We support all browser versions ${B} or above and will only require browser features in ${F}.

This feels reasonable to me on the surface. However it gets tricky if a new browser is added and regresses the feature set. In theory, any previously released versions of a library/framework should be isolated from that change, given that they only committed to supporting that "widely available" Baseline at their release date (browsers ${B} and features ${F}).

So my ask for the Baseline definition would be to version it in a similar fashion. Say a new browser is added to the core set on date ${D}. The exact semantics on this are undefined and TBD as you mentioned, but I'm wondering if Baseline could at least commit to not changing the definition of ${B} and ${F} for date ${D - 1} (and earlier). If a new browser is introduced, it should only affect Baseline at some particular start date and the dates after, but no dates before. This allows versioned tools to rely on a stable definition of Baseline at a particular date.

This would still give you the freedom to dictate exactly when and how a new browser is added while allowing the ecosystem to use Baseline without fear of a new browser addition undermining their existing released versions. I expect this might be a more academic concern as in practice I don't expect a large proportion of users would file bugs against their libraries/frameworks/tools saying "Please add support for this newly included browser to all your existing released versions because Baseline has been updated and that retroactively applies to all your prior releases." Most maintainers are just going to close those kinds of issues, but it would be nice if the precise wording of such a browser policy did match the way most humans would be likely to interpret it.