Feature questions: Skip "used it" for features with no implementations

[LeaVerou]

Benefits:

• 33% fewer options, lower cognitive overhead
• Lower scrolling distance
• It's a signal that the feature doesn't have implementations, while currently asking if they've used a feature they could not possibly have used is a little disingenuous (we've seen expert users getting thrown because of this)

Drawbacks:

• Some implementation effort, though hopefully very small
• Sometimes people have used unimplemented features via polyfills (though whether this counts is open to interpretation, so data quality is poor)
• It's after the user testing, so we can't know if people will be puzzled ("why is there no 'used it' option?")
• Users will learn to expect it, which means if a feature question doesn't have this flag, it signifies that it is implemented.

[SachaG]

I'm not in favor of this because:

1. it's one more thing to keep track of, so one more thing that can go wrong.
2. It hurts consistency
3. With the State of CSS, I think we concluded that even for non-implemented features, people might've used them with polyfills or other workarounds so we wouldn't remove the option.

[LeaVerou]

1. it's one more thing to keep track of, so one more thing that can go wrong.

Doesn't this apply to everything?

2. It hurts consistency

One way to be more consistent would be to still show the option, but disabled, with a "Not implemented anywhere yet" tooltip. That makes it far less confusing, but doesn't decrease scrolling distance (likely a worthy tradeoff).

3. With the State of CSS, I think we concluded that even for non-implemented features, people might've used them with polyfills or other workarounds so we wouldn't remove the option.

Yes, I made the same point in my first post. Although, it is not at all universally accepted that this would qualify for "used it" (in the user study we even had people who were unsure whether to select "used it" because they hadn't used a feature on an actual production website!), so it could be argued that it increases noise in the data if some polyfill users select "used it" and others "heard of it".

[SachaG]

What I would like to eventually have to address this issue is a better integration with CanIUse (or just some manual metadata) that would let us show what "stage" any feature is at. So something like "fully implemented", "partial browser support", "experimental support", "proposal"… I think this would be more informative than just disabling one option for some of the features.

[LeaVerou]

Agreed, but since we can't do that right now, it seems like the binary distinction "implemented at all" vs "not implemented anywhere" may be more useful than nothing — and is easier to maintain.

Would you actively veto this (either disabling the 3rd option or hiding it — I'm fine either way, and open to other ideas too), or just not motivated enough to implement it yourself? If the latter, I could try to take a stab.

[LeaVerou]

After observing one more subject fill in the survey, I'm convinced we should do something about this. Based on what I've observed, I'm almost certain most of the "used it"s we get from these features are not polyfill usage, but simply respondents getting confused and mistaking the features for different features that they have, actually, used (e.g. looks at HTML Modules, "oh, I've used modules").

I have a few ideas about what we could do:

1. Hide "Used it" entirely.
   • Pros: reduces scrolling, simplifies question format.
   • Cons: confusing ("why do some features not have a used it option?") though that can be mitigated by also marking that this is a proposal (see No 5 below)
2. Disable "Used it" with a tooltip that says something like "Not implemented".
   • Pros: more clear
   • Cons: less complexity reduction, same length
3. Change "Used it" to "Used it (via a polyfill)". Might need different wording to also encompass preprocessors and other scripts.
   • Pros: those who have used it via a polyfill can still select it, but it's a clear signal to those who haven't that this is not part of the platform yet
   • Cons: Smaller reduction in cognitive overhead, as there are still 3 options, could still potentially get ignored
4. Show browser support icons next to each feature
   • Pros: Communicates even more information
   • Cons: Way more effort to implement, more data to maintain
5. Mark certain features as "Unimplemented proposal". This can also be combined with 1, 2, 3 above.
   • Pros: Does not interfere with answers or data model
   • Cons: May still be ignored, esp. depending on styling

In descending order of preference, my recommendation would be 2 or 3, 4, 5 + 1, 5

In this survey, these are the entirely unimplemented full feature questions:

• <selectlist>
• Accordion
• focusgroup
• DOM Parts
• <model>
• HTML Modules
• Scoped custom element registries
• Isolated Web Apps

[SachaG]

Disable "Used it" with a tooltip that says something like "Not implemented".

This seems like the best solution to me since it informs the user about the state of the feature, while not being too disruptive. Let's hardcode it for now, and we can think about the best API for it later.

[SachaG]

What would the text of the tooltip be?

[SachaG]

[SachaG]

Actually I think No browser implementation yet is less ambiguous.