Research: detecting polyfills

[43081j]

If we can land #257 , it means we can better represent polyfills since we can specify their target platform/engine versions.

We should do a big push to list as many widely used polyfills in this repo, along with their engine constraints and their replacements.

For example:

• Polyfill: abort-controller
• Support: web features doc

So we would want to define this similar to the following:

{
  "moduleName": "abort-controller",
  "replacements: [
    {
      "type": "native",
      "replacement": "AbortController",
      "engines": [
        {
          "type": "baseline",
          "category": "aborting",
          "key": "api.AbortController"
        },
        {
          "type": "engine",
          "name": "node",
          "version": ">=18.0.0", // put real version here
        }
      },
  ]
}

this should help us define #257 as we add more polyfills to this issue

in this example, we can use the category and key to access the web-features npm package's data set to get hold of the browser versions.

we still need to define node, bun, etc ourselves.

so tasks for us to do:

• list as many popular polyfills as we can that are redundant in modern browsers/engines
• help bikeshed the data shape for Repo: next major version of schema #257

[arcaniussainey]

I was curious regarding the engine specific APIs. Particularly with this:

"engine‑specific APIs are the tricky bit. For Node we at least have a machine‑readable source: https://nodejs.org/api/all.html and its JSON twin https://nodejs.org/api/all.json, so built‑ins (like styleText and friends) are parseable. The trouble is Bun and Deno: they don’t expose a "since/added" info in their docs. "

If we want a machine readable source, could we not use the machine readable source for node and test what's supported there, and then inspect the methods of the global object? And just target that at different versions to produce a list for each version? I haven't actually attempted it yet, but since Bun/Deno provide their APIs through a global object that should be doable no?

Then to get a list of engine API features we'd just need a list of the global objects to check for each engine?

Also regarding checking for polyfills, both modules and syntax checks were mentioned. With respect to syntax checks, are you thinking like a bag of fingerprints? Because in that case we could use a list of CTPHs of something like the unoptimized and optimized tokens for a function to see how similar it is to a known Polyfill. It'd be slower than assuming based on imports but should still be tractable.

That said, as I see the present tasks we're trying to create: a collection of samples, a collection of import rules, and a way of linking them. This seems the same as malware detection in principle and function. Have we already looked at how virus and anti-malware DBs handle issues like this with JavaScript? If not, that could be a good ref.

However, I fear I may be misunderstanding things though, I'm new here. Lmk if I'm well off base, and apologies for not including any actual schema change suggestions in this comment. I wanted to know if I was completely off base 😭.

[43081j]

much of this will be static analysis. usually we will have just have access to the package.json and sources. not necessarily a runtime or even node_modules existing.

that means we will be analysing package.json for browserlist and engines, which we can push people to make more use of in the wild.

this is how the existing ESLint plugin and CLI work:

• find all imports in files being analysed
• compare to the module-replacements data set
• if any match, show the suggested replacement
• optionally, if a codemod exists, auto-fix the migration

this doesn't work well for polyfills purely because we don't take the target engines into account.

so the missing piece here is:

• find all imports in files being analysed
• compare to the module-replacements data set
  • compare engines/browserlist against the replacements' constraints
• if any match, show the suggested replacement
• optionally, if a codemod exists, auto-fix the migration

that new piece would basically consist of comparing the engines array of a replacement to the context we're in, which we will have built up based on the user's browserlist/engines in their package.json.

we want to avoid duplicating the data for what versions added support for various features, so that's where the web-features package and maybe node's JSON come in.

[zloirock]

You could take a look at unicorn/no-unnecessary-polyfills that's based on core-js-compat.

[43081j]

core-js-compat could be useful for sure

i think ultimately we will end up wanting to combine a few different data sets. so we probably want to combine the corejs compat data at build time or something. needs some thought, but we should make use of that as it already defines all the corejs ones at least

[zloirock]

As alternative data sources, you can consider mdn/browser-compat-data, caniuse, and compat-table. However, experience has shown that they usually do not take into account bugs and are poorly designed to work with such tools, so you should be careful with them.