[Incorrect Data] Usage data for some features is suspect

[rmarx]

Hello there,

I've been ingesting some of the webstatus.dev API for use on the RUM Archive insights page (https://rumarchive.com/insights/). In the "Baseline" section, we allow people to toggle showing the daily Chrome usage stats for each feature.

However, we've noticed some unexpected usage numbers for some features. One of them was very obvious: the popover API, which used to show it was used on 50%+ pages. However, that was due to a bug where <select> was being counted as popover and that has recently been fixed.

There are others we've detected though:

• clip-path showing as 40% of all pages (here) and Masks on 34.1% of pages (here): these seem high for what I consider to be somewhat niche features?
• Offscreen canvas at 6.5% (here) seems a bit high? Even when considering fingerprinting libraries? Same for OfflineAudioContext at 5.2% (here)
• JavaScript Modules at 28.5% (here): quite a contrast with Web Almanac data putting that at a mere 1.2% last year
• Forced Colors at 32.2% (here) seems high (would be a great win for accessibility though 😉)
• 2024 Backdrop Filter already at 24.8% (while Baseline 2017’s Grid is just on 36.7%? That feels wrong 😊)

There are others, but the gist is clear... I can't be sure of course, but it seems like there are issues with the chrome usage statistics for several of these features.

I'm not sure this is the right repo to report this on, but it's unclear to me where Chrome daily usage data is coming from... it is present in the baseline API from webstatus.dev (e.g., in https://api.webstatus.dev/v1/features?q=baseline_date%3A2021-01-01..2021-12-31), but it's not coming from one of the sources listed in the issue template (e.g., web-platform-dx/web-features, or I'm missing something). Barry Pollard recommended I open the issue here.

If there is a separate repository where this issue would be more at home, I'd be happy to re-open it there.

Thank you in advance for taking a look!

[jcscottiii]

Thanks for this valuable report!

Here are some thoughts:

Context on the Data

The popover situation you mentioned is the perfect example. To give more specific context on what happened there, the original mojom entry (kPopover = 3) was deprecated and became kObsolete_Popover = 3, while a new, correct entry was added (kPopover = 374).

Our pipeline was not designed to automatically detect this "value swapping" scenario. We have since manually corrected our ingestion for popover and any other features that followed this pattern. This is why we are confident that the high numbers you're currently seeing are not a lingering ingestion bug on our end.

(We are also nearly finished with an automated, long-term fix to handle this specific deprecation scenario correctly from the start.)

Verification

I've verified that the numbers you're seeing are exactly what we are receiving from the upstream data source (Chrome's UMA). They are consistent with the data shown on ChromeStatus.com, which consumes from the same source.

• clip-path: Similar to ChromeStatus.
• masks: Similar to ChromeStatus.
• offscreen-canvas: Similar to ChromeStatus.
• offline-audio-context: Similar to ChromeStatus.
• js-modules: Similar to ChromeStatus.
• forced-colors: Similar to ChromeStatus.
• backdrop-filter: Similar to ChromeStatus.

(Note: ChromeStatus shows the monthly usage in the chart. But if you examine their API, it shows the same daily metrics)

This consistency confirms that we are accurately ingesting and displaying the data provided to us.

Next Steps

If there is a potential issue with the source data itself, the correct way to get it investigated is to file a report against Chromium, so the team owning those metrics can look into the source.

The best place to file this would be on the [Chromium Bug Tracker. It would be very helpful to link back to this GitHub issue and copy the details from your report.

cc: @atopal