Skip to content

test: improve coverage and strictness#2193

Open
kheiner wants to merge 27 commits intonuxt:mainfrom
kheiner:refactor(playground)/alphebatize-providers
Open

test: improve coverage and strictness#2193
kheiner wants to merge 27 commits intonuxt:mainfrom
kheiner:refactor(playground)/alphebatize-providers

Conversation

@kheiner
Copy link
Copy Markdown
Contributor

@kheiner kheiner commented Mar 27, 2026

The providers are now organized alphabetically. I also updated the Directus example in the playground to reflect a public dataset and public image src.

🔗 Linked issue

resolves #2192

📚 Description

Providers were not sorted in any way, so finding your provider in the list was not intuitive.

Additional: Previously the Directus provider pointed to a localhost:8055/assets url that was unique to each person, ensuring that the 'src' could not be duplicated for testing.

…ctus baseURL.

Updated the Directus URL with a publicly available example.
@kheiner kheiner requested a review from danielroe as a code owner March 27, 2026 22:31
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Mar 27, 2026

Open in StackBlitz

npm i https://pkg.pr.new/@nuxt/image@2193

commit: 6796cf6

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 27, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Reworked the playground providers dataset and related test fixtures: playground/app/providers.ts was extensively reordered and had many provider entries added, removed, or reconfigured (notably caisy, cloudimage, directus, sirv, supabase, imagekit, imgix, san ity, storyblok, twicpics, vercel and others) with updated sample srcs, modifiers, sizing, formats and operations. playground/nuxt.config.ts changed directus.baseURL to https://example.directus.app/assets/. Multiple e2e snapshots (directus, caisy, fastly) and test fixtures were updated. Tests were reworked (test/nuxt/providers.test.ts, test/providers.ts, test/unit/provider-coverage.test.ts) and runtime GitHub sizing logic (src/runtime/providers/github.ts) was adjusted.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description is clearly related to the changeset, explaining that providers are now alphabetically organized and Directus was updated to use a public dataset instead of localhost.
Linked Issues check ✅ Passed The PR successfully addresses all objectives from issue #2192: providers are alphabetically organized, provider-coverage tests validate runtime/playground consistency, and missing provider tests are tracked.
Out of Scope Changes check ✅ Passed All changes are directly in scope: playground provider reorganization, test coverage improvements, GitHub provider fixes, Directus configuration updates, and documentation for GitHub provider sizing.
Title check ✅ Passed The title 'test: improve coverage and strictness' accurately captures the main focus of the PR, which adds comprehensive provider coverage tests, reorganizes providers alphabetically, and improves test strictness. It aligns well with the stated objectives and the substantial changes to test files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 32.52%. Comparing base (7e561f6) to head (6796cf6).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2193   +/-   ##
=======================================
  Coverage   32.52%   32.52%           
=======================================
  Files           7        7           
  Lines         372      372           
  Branches      131      131           
=======================================
  Hits          121      121           
  Misses        194      194           
  Partials       57       57           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

kheiner added 2 commits March 27, 2026 15:57
Coderabbit noticed that Strapi5 was not retained in
Strapi5 is not a duplicate, was omitted by accident.
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
playground/app/providers.ts (1)

852-978: ⚠️ Potential issue | 🟡 Minor

Alphabetical ordering issue: storyblok should come before strapi.

The PR objective is to alphabetize providers, but storyblok (s-t-o) is incorrectly placed after strapi (s-t-r). Alphabetically, "o" comes before "r", so the correct order should be:

sirvstoryblokstrapistrapi5supabase

Suggested reordering

Move the storyblok provider block (lines 912-978) to appear before the strapi provider block (line 852).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@playground/app/providers.ts` around lines 852 - 978, The providers are out of
alphabetical order: move the entire provider object whose name is 'storyblok' so
it appears before the provider object named 'strapi' (keeping 'strapi5' after
'strapi'), i.e., reorder the provider blocks so the sequence is ... 'sirv',
'storyblok', 'strapi', 'strapi5', 'supabase'; ensure you cut and paste the full
'storyblok' samples block (including its sample objects and trailing commas) and
place it immediately before the 'strapi' block without altering the inner sample
data.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@playground/app/providers.ts`:
- Around line 852-978: The providers are out of alphabetical order: move the
entire provider object whose name is 'storyblok' so it appears before the
provider object named 'strapi' (keeping 'strapi5' after 'strapi'), i.e., reorder
the provider blocks so the sequence is ... 'sirv', 'storyblok', 'strapi',
'strapi5', 'supabase'; ensure you cut and paste the full 'storyblok' samples
block (including its sample objects and trailing commas) and place it
immediately before the 'strapi' block without altering the inner sample data.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cd491017-5729-4108-ba16-86ac05fe6772

📥 Commits

Reviewing files that changed from the base of the PR and between c942090 and 65a4583.

📒 Files selected for processing (1)
  • playground/app/providers.ts

Fixed sort of strap/strapi5/storyblok
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@playground/app/providers.ts`:
- Around line 338-368: The Directus provider configuration named 'directus'
contains sample entries under samples with UUIDs in the src fields but the
provider's baseURL is still the placeholder
https://example.directus.app/assets/, so image requests will fail; update the
provider configuration for 'directus' to point baseURL to a real public Directus
instance (or document the need to replace it), or replace the sample src values
with fully-qualified URLs that resolve against your actual Directus baseURL so
the samples in samples (src: '39ff62fd-902e-458c-808f-702460c51309', etc.) will
load correctly.
- Around line 266-273: Update the image src string in
playground/app/providers.ts to remove the unnecessary token query parameter from
the GitBook URL; locate the object literal (the entry with keys src, width,
height, densities, fit, quality, format) and replace the src value so it ends at
.../image.png (no ?alt=media&token=... suffix) to use the public URL without
authentication.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d36c4176-cc88-4edb-9f03-e355dc835df7

📥 Commits

Reviewing files that changed from the base of the PR and between 65a4583 and 1ea1827.

📒 Files selected for processing (1)
  • playground/app/providers.ts

@kheiner kheiner changed the title refactor(playground): Organize providers alphabetically refactor(playground): Organize providers alphabetically in tests and playground Mar 28, 2026
Add comments inline for providers missing tests.

Add and comment imports for missing providers.
@kheiner kheiner marked this pull request as draft March 28, 2026 15:18
@kheiner
Copy link
Copy Markdown
Contributor Author

kheiner commented Mar 28, 2026

Converting to draft to increase scope in fixing some of the missing/faked tests.

@kheiner kheiner changed the title refactor(playground): Organize providers alphabetically in tests and playground fix(test,playground): improve test coverage and strictness Mar 28, 2026
@kheiner
Copy link
Copy Markdown
Contributor Author

kheiner commented Mar 29, 2026

@coderabbitai come back and do another sweep - I've got this ready for review.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 29, 2026

@kheiner Sure thing! I'll kick off a fresh review sweep now.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@kheiner kheiner marked this pull request as ready for review March 29, 2026 06:09
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/unit/provider-coverage.test.ts (1)

7-18: ⚠️ Potential issue | 🟠 Major

Separate playground existence checks from matrix coverage exclusions.

missingProviderTests is now being used for two different things: “providers excluded from the shared images matrix” and “providers still missing runtime coverage”. That drops strapi/strapi5 from the exact-once playground assertion, and the inline snapshot at Lines 93-95 is counting exclusions instead of the real TODO runtime tests—even though test/nuxt/providers.test.ts still leaves Bunny and Caisy as it.todo.

🧭 Suggested split
-const missingProviderTests = [
+const matrixCoverageExclusions = [
   'ipxStatic', // build-time-only alias for ipx
   'strapi', // covered in a unique test
   'strapi5', // covered in a unique test
 ]
+const todoProviderTests = ['bunny', 'caisy']
 
 const providers = files.map(f => f.replace(/\.ts$/, ''))
-const withProviderTests = providers.filter(p => !missingProviderTests.includes(p))
+const providersExpectedInPlayground = providers.filter(p => p !== 'ipxStatic')
+const providersWithMatrixFixtures = providers.filter(p => !matrixCoverageExclusions.includes(p))
 
-const matrixCases = withProviderTests.flatMap(provider =>
+const matrixCases = providersWithMatrixFixtures.flatMap(provider =>
   images.map((image, index) => ({
     provider,
     image,
     index,
   })),
 )
@@
   it('each provider should exist exactly once in playground', () => {
-    for (const provider of withProviderTests) {
+    for (const provider of providersExpectedInPlayground) {
       const matches = playgroundProviders.filter(p => p.name === provider)
       expect(
         matches.length,
         `expect exactly 1 match for "${provider}", got ${matches.length}`,
       ).toBe(1)
     }
   })
@@
-    const allowedKeys = new Set(['args', ...withProviderTests])
+    const allowedKeys = new Set(['args', ...providersWithMatrixFixtures])
@@
-    for (const provider of withProviderTests) {
+    for (const provider of providersWithMatrixFixtures) {
       const count = images.filter(img =>
         Object.prototype.hasOwnProperty.call(img, provider),
       ).length
@@
-  it('expect missingProviderTests to shrink over time', () =>
+  it('expect todo provider tests to shrink over time', () =>
     // TODO: remove this once all providers have tests
-    expect(missingProviderTests.length).toMatchInlineSnapshot(`3`))
+    expect(todoProviderTests).toMatchInlineSnapshot(`
+      [
+        "bunny",
+        "caisy",
+      ]
+    `))

Also applies to: 35-41, 93-95

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/unit/provider-coverage.test.ts` around lines 7 - 18, The current
missingProviderTests array is being used for two different purposes
(playground/matrix exclusions and runtime-coverage TODOs); split it into two
distinct arrays (e.g., playgroundExcludedProviders and runtimeMissingProviders),
update usages so providers.filter(...) uses playgroundExcludedProviders to build
withProviderTests, and update the exact-once playground assertion and the
snapshot/TODO counting to reference runtimeMissingProviders instead; ensure
unique symbols referenced are missingProviderTests (to remove),
withProviderTests (to keep deriving from playgroundExcludedProviders), and any
assertions around the inline snapshot or it.todo checks to use
runtimeMissingProviders.
🧹 Nitpick comments (1)
test/nuxt/providers.test.ts (1)

319-325: Add a boundary test for the new GitHub size rules.

This block only covers the default and 200px paths from the shared matrix. The newly documented cap/fallback behavior—at least the width: 512 -> s=460 case—can still regress without a dedicated assertion here.

➕ Suggested test
   it('github', () => {
     const providerOptions = {}
     for (const image of images) {
       const [src, modifiers] = image.args
       const generated = github().getImage(src, { modifiers, ...providerOptions }, getEmptyContext())
       expect(generated).toMatchObject(image.github)
     }
   })
+
+  it('github caps oversized avatars', () => {
+    const generated = github().getImage('unjs', { modifiers: { width: 512 } }, getEmptyContext())
+    expect(generated).toMatchObject({
+      url: 'https://avatars.githubusercontent.com/unjs?v=4&s=460',
+    })
+  })
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/nuxt/providers.test.ts` around lines 319 - 325, The current 'github'
test iterates shared images but misses a boundary assertion for the new GitHub
size rules; add an explicit test case (either inside the existing it('github')
loop or as a new it block) that calls github().getImage with a src/args
representing width:512 (use the same images/test helpers) and assert the
produced URL or modifiers include the expected cap/fallback (e.g. s=460) to
prevent regressions; reference github(), getImage, images and the 'github' test
name when locating where to add the assertion.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/content/3.providers/github.md`:
- Line 22: The inline explanatory comment placed after the NuxtImg component
(<NuxtImg provider="github" src="unjs" width="512" />) uses JavaScript-style //
which is invalid in a Vue template; move that explanatory note out of the
component tag into surrounding prose or convert it to an HTML comment (<!-- ...
-->) so the example remains valid SFC template code and a reader can copy-paste
without errors.

In `@src/runtime/providers/github.ts`:
- Around line 18-20: The current avatar sizing uses the nullish coalescing (??
1) which still allows zero/negative values to pass through: update the sizing
logic in the GitHub provider where you check modifiers?.width /
modifiers?.height and set size (the block with if (modifiers?.width ||
modifiers?.height) and size = Math.min(...)). Fix by coercing modifiers?.width
and modifiers?.height to numbers, treating non-positive values as absent, taking
the numeric max of width and height, then clamping that result to the inclusive
range 1..460 and assigning it to size.

---

Outside diff comments:
In `@test/unit/provider-coverage.test.ts`:
- Around line 7-18: The current missingProviderTests array is being used for two
different purposes (playground/matrix exclusions and runtime-coverage TODOs);
split it into two distinct arrays (e.g., playgroundExcludedProviders and
runtimeMissingProviders), update usages so providers.filter(...) uses
playgroundExcludedProviders to build withProviderTests, and update the
exact-once playground assertion and the snapshot/TODO counting to reference
runtimeMissingProviders instead; ensure unique symbols referenced are
missingProviderTests (to remove), withProviderTests (to keep deriving from
playgroundExcludedProviders), and any assertions around the inline snapshot or
it.todo checks to use runtimeMissingProviders.

---

Nitpick comments:
In `@test/nuxt/providers.test.ts`:
- Around line 319-325: The current 'github' test iterates shared images but
misses a boundary assertion for the new GitHub size rules; add an explicit test
case (either inside the existing it('github') loop or as a new it block) that
calls github().getImage with a src/args representing width:512 (use the same
images/test helpers) and assert the produced URL or modifiers include the
expected cap/fallback (e.g. s=460) to prevent regressions; reference github(),
getImage, images and the 'github' test name when locating where to add the
assertion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 24970146-6a0f-4083-aad7-15c032d0caf2

📥 Commits

Reviewing files that changed from the base of the PR and between 65a4583 and 29da6db.

📒 Files selected for processing (8)
  • docs/content/3.providers/github.md
  • playground/app/providers.ts
  • src/runtime/providers/github.ts
  • test/e2e/__snapshots__/caisy.json5
  • test/e2e/__snapshots__/fastly.json5
  • test/nuxt/providers.test.ts
  • test/providers.ts
  • test/unit/provider-coverage.test.ts
✅ Files skipped from review due to trivial changes (2)
  • test/e2e/snapshots/caisy.json5
  • test/e2e/snapshots/fastly.json5

@kheiner
Copy link
Copy Markdown
Contributor Author

kheiner commented Mar 29, 2026

test/unit/provider-coverage.test.ts (1)> 7-18: ⚠️ Potential issue | 🟠 Major

Separate playground existence checks from matrix coverage exclusions.
missingProviderTests is now being used for two different things: “providers excluded from the shared images matrix” and “providers still missing runtime coverage”. That drops strapi/strapi5 from the exact-once playground assertion, and the inline snapshot at Lines 93-95 is counting exclusions instead of the real TODO runtime tests—even though test/nuxt/providers.test.ts still leaves Bunny and Caisy as it.todo.

🧭 Suggested split

-const missingProviderTests = [
+const matrixCoverageExclusions = [
   'ipxStatic', // build-time-only alias for ipx
   'strapi', // covered in a unique test
   'strapi5', // covered in a unique test
 ]
+const todoProviderTests = ['bunny', 'caisy']
 
 const providers = files.map(f => f.replace(/\.ts$/, ''))
-const withProviderTests = providers.filter(p => !missingProviderTests.includes(p))
+const providersExpectedInPlayground = providers.filter(p => p !== 'ipxStatic')
+const providersWithMatrixFixtures = providers.filter(p => !matrixCoverageExclusions.includes(p))
 
-const matrixCases = withProviderTests.flatMap(provider =>
+const matrixCases = providersWithMatrixFixtures.flatMap(provider =>
   images.map((image, index) => ({
     provider,
     image,
     index,
   })),
 )
@@
   it('each provider should exist exactly once in playground', () => {
-    for (const provider of withProviderTests) {
+    for (const provider of providersExpectedInPlayground) {
       const matches = playgroundProviders.filter(p => p.name === provider)
       expect(
         matches.length,
         `expect exactly 1 match for "${provider}", got ${matches.length}`,
       ).toBe(1)
     }
   })
@@
-    const allowedKeys = new Set(['args', ...withProviderTests])
+    const allowedKeys = new Set(['args', ...providersWithMatrixFixtures])
@@
-    for (const provider of withProviderTests) {
+    for (const provider of providersWithMatrixFixtures) {
       const count = images.filter(img =>
         Object.prototype.hasOwnProperty.call(img, provider),
       ).length
@@
-  it('expect missingProviderTests to shrink over time', () =>
+  it('expect todo provider tests to shrink over time', () =>
     // TODO: remove this once all providers have tests
-    expect(missingProviderTests.length).toMatchInlineSnapshot(`3`))
+    expect(todoProviderTests).toMatchInlineSnapshot(`
+      [
+        "bunny",
+        "caisy",
+      ]
+    `))

Also applies to: 35-41, 93-95

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@test/unit/provider-coverage.test.ts` around lines 7 - 18, The current
missingProviderTests array is being used for two different purposes
(playground/matrix exclusions and runtime-coverage TODOs); split it into two
distinct arrays (e.g., playgroundExcludedProviders and runtimeMissingProviders),
update usages so providers.filter(...) uses playgroundExcludedProviders to build
withProviderTests, and update the exact-once playground assertion and the
snapshot/TODO counting to reference runtimeMissingProviders instead; ensure
unique symbols referenced are missingProviderTests (to remove),
withProviderTests (to keep deriving from playgroundExcludedProviders), and any
assertions around the inline snapshot or it.todo checks to use
runtimeMissingProviders.

I think code rabbit is really taking issue with my naming and not with the function of the tests. I'm happy to look into making the changes as it's suggesting but for now I'm hoping someone else can offer an opinion.

…alphebatize-providers

# Conflicts:
#	playground/app/providers.ts
#	test/nuxt/providers.test.ts
#	test/providers.ts
@danielroe danielroe changed the title fix(test,playground): improve test coverage and strictness fix(github): set max size - & improve test coverage/strictness Apr 1, 2026
@danielroe danielroe changed the title fix(github): set max size - & improve test coverage/strictness test: improve coverage and strictness Apr 3, 2026
@kheiner
Copy link
Copy Markdown
Contributor Author

kheiner commented Apr 4, 2026

it looks like this PR was only partially merged into main. Can I get clarification on what the plan is forward? I'm fine with being told the remainder of the changes do not align or aren't accepted;

My original PR description did not fully explain what I did (my mistake, hopefully this clears it up):

The changes to test/unit/provider-coverage.test.ts were intended to keep contributions clean and within a uniform format:

  • test/providers.ts should contain only the provider using the name of the provider exactly once, and additionally not allow unknown entries.
    • Addresses: prismic and prismicUnsplash cases to test, but only prismic is a provider.
    • Addresses: fastly and fastlyAbsoluteUrl cases to test but only fastly is a provider.
  • ensure that the test/providers.ts image[] has a key for each provider for every index in the array.
    • Addresses: github is only present in the 0 and 1 index of the images array.
  • ensure each provider exists exactly once in playground
    • Addresses: netlifyImageCdn and netlifyLargeMedia are missing in the playground
  • ensure all playground providers exist in runtime (not currently a problem but it could be)

There were also quite a few integration tests that did not appropriately consider the 'src' and/or were missing from test/nuxt/providers.test.ts

  • bunny, caisy, directus, filerobot, github missing.
  • sanity, prepr ignore image.args.src in favor of manually creating their own image src that doesn't actually have meaning to the test

if I were to do it all over again I might choose to make commits that separate the movement alphabetically into commented blocks and then delete them so it's easier to understand/follow the changes. Let me know if this helps or if I'm off base. If these changes aren't wanted/needed I'd like to delete the branch and close the PR.

@kheiner kheiner requested a review from danielroe April 4, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add comprehensive provider coverage tests to ensure runtime and playground consistency.

3 participants