Always include Vary: Accept-Encoding for compressible responses

[kubaflo]

🤖 AI Summary

🔍 Automated Fix Report

---

🔍 Pre-Flight — Context & Validation

Issue: #48008 — Response compression middleware doesn't add Vary: Accept-Encoding header for compressible responses

Area: src/Middleware/ResponseCompression/

Root Cause: The middleware only wraps the response body (and adds Vary: Accept-Encoding) when the request contains an Accept-Encoding header. Responses to requests without Accept-Encoding don't get the Vary header, which causes CDNs/proxies to cache uncompressed responses and serve them to clients that do support compression.

Classification: Bug — missing HTTP caching directive

---

🧪 Test — Bug Reproduction

Test File: src/Middleware/ResponseCompression/test/ResponseCompressionMiddlewareTest.cs

Tests Verified: Existing tests cover Vary header behavior. New test verifies Vary: Accept-Encoding is present even when the request lacks Accept-Encoding.

Strategy: Verified that requests without Accept-Encoding to compressible content types still receive Vary: Accept-Encoding in the response.

---

🚦 Gate — Test Verification & Regression

Gate Result: ✅ All 141 ResponseCompression tests pass

Test Command:

dotnet test src/Middleware/ResponseCompression/test/Microsoft.AspNetCore.ResponseCompression.Tests.csproj --no-restore -v q

Regression: No failures in existing test suite.

---

🔧 Fix — Analysis & Comparison (✅ 2 passed, ❌ 1 failed)

Fix: Always add Vary: Accept-Encoding for compressible content types, but only compress when request has Accept-Encoding.

Attempt	Approach	Result
0	Always wrap body, separate allowCompression field	✅ Pass
1	OnStarting callback for Vary header	❌ Fail (ShouldCompressResponse internal)
2	Early return in InitializeCompressionHeaders after Vary	✅ Pass

✅ Attempt 0: PASS

Approach: Always wrap response body with allowCompression flag; add Vary header for compressible types regardless of Accept-Encoding.

Modified ResponseCompressionMiddleware.Invoke to always call InvokeCore. Added _allowCompression field to ResponseCompressionBody — set false when no Accept-Encoding. Vary header added for compressible content types regardless, but actual compression only happens when _allowCompression is true.

📄 Diff

diff --git a/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs b/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
index 2eefdb6482..ef74ba8916 100644
--- a/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
+++ b/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
@@ -26,14 +26,16 @@ internal sealed class ResponseCompressionBody : Stream, IHttpResponseBodyFeature
     private bool _providerCreated;
     private bool _autoFlush;
     private bool _complete;
+    private readonly bool _allowCompression;
 
     internal ResponseCompressionBody(HttpContext context, IResponseCompressionProvider provider,
-        IHttpResponseBodyFeature innerBodyFeature)
+        IHttpResponseBodyFeature innerBodyFeature, bool allowCompression)
     {
         _context = context;
         _provider = provider;
         _innerBodyFeature = innerBodyFeature;
         _innerStream = innerBodyFeature.Stream;
+        _allowCompression = allowCompression;
     }
 
     internal async Task FinishCompressionAsync()
@@ -224,17 +226,22 @@ internal sealed class ResponseCompressionBody : Stream, IHttpResponseBodyFeature
                 headers.Vary = StringValues.Concat(headers.Vary, HeaderNames.AcceptEncoding);
             }
 
-            var compressionProvider = ResolveCompressionProvider();
-            if (compressionProvider != null)
+            // Only attempt compression when the request includes Accept-Encoding.
+            // The Vary header is always added above for correct caching behavior.
+            if (_allowCompression)
             {
-                // Can't use += as StringValues does not override operator+
-                // and the implict conversions will cause an incorrect string concat https://github.com/dotnet/runtime/issues/52507
-                headers.ContentEncoding = StringValues.Concat(headers.ContentEncoding, compressionProvider.EncodingName);
-                headers.ContentMD5 = default; // Reset the MD5 because the content changed.
-                headers.ContentLength = default;
-            }
+                var compressionProvider = ResolveCompressionProvider();
+                if (compressionProvider != null)
+                {
+                    // Can't use += as StringValues does not override operator+
+                    // and the implict conversions will cause an incorrect string concat https://github.com/dotnet/runtime/issues/52507
+                    headers.ContentEncoding = StringValues.Concat(headers.ContentEncoding, compressionProvider.EncodingName);
+                    headers.ContentMD5 = default; // Reset the MD5 because the content changed.
+                    headers.ContentLength = default;
+                }
 
-            return compressionProvider;
+                return compressionProvider;
+            }
         }
 
         return null;
diff --git a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
index 22a9677d6e..4a67f0418f 100644
--- a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
+++ b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
@@ -36,21 +36,17 @@ public class ResponseCompressionMiddleware
     /// <returns>A task that represents the execution of this middleware.</returns>
     public Task Invoke(HttpContext context)
     {
-        if (!_provider.CheckRequestAcceptsCompression(context))
-        {
-            return _next(context);
-        }
-        return InvokeCore(context);
+        return InvokeCore(context, _provider.CheckRequestAcceptsCompression(context));
     }
 
-    private async Task InvokeCore(HttpContext context)
+    private async Task InvokeCore(HttpContext context, bool allowCompression)
     {
         var originalBodyFeature = context.Features.Get<IHttpResponseBodyFeature>();
         var originalCompressionFeature = context.Features.Get<IHttpsCompressionFeature>();
 
         Debug.Assert(originalBodyFeature != null);
 
-        var compressionBody = new ResponseCompressionBody(context, _provider, originalBodyFeature);
+        var compressionBody = new ResponseCompressionBody(context, _provider, originalBodyFeature, allowCompression);
         context.Features.Set<IHttpResponseBodyFeature>(compressionBody);
         context.Features.Set<IHttpsCompressionFeature>(compressionBody);
 

❌ Attempt 1: FAIL

Approach: Use Response.OnStarting callback to add Vary header when not going through compression path.

Register an OnStarting callback in the non-compression path that checks ShouldCompressResponse and adds Vary: Accept-Encoding. This avoids wrapping the body. Failed because ShouldCompressResponse is an internal method and the approach requires content-type inspection that isn't available at middleware dispatch time.

📄 Diff

diff --git a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
index 22a9677d6e..1eb3f9381c 100644
--- a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
+++ b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
@@ -38,6 +38,16 @@ public class ResponseCompressionMiddleware
     {
         if (!_provider.CheckRequestAcceptsCompression(context))
         {
+            // Even when not compressing, add Vary: Accept-Encoding for compressible content
+            // so CDNs/proxies know the response varies by Accept-Encoding
+            context.Response.OnStarting(() =>
+            {
+                if (_provider.ShouldCompressResponse(context))
+                {
+                    context.Response.Headers.Append(Microsoft.Net.Http.Headers.HeaderNames.Vary, "Accept-Encoding");
+                }
+                return Task.CompletedTask;
+            });
             return _next(context);
         }
         return InvokeCore(context);

✅ Attempt 2: PASS

Approach: Pass allowCompression flag through middleware to body, add early return in InitializeCompressionHeaders after Vary header.

Modified Invoke to always call InvokeCore(context, allowCompression). ResponseCompressionBody accepts allowCompression parameter. In InitializeCompressionHeaders, always add Vary header for compressible types, but return null (skip compression) when !_allowCompression. Same logical approach as attempt 0 but structured differently — the flag check is inside the compression headers method rather than affecting provider resolution.

📄 Diff

diff --git a/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs b/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
index 2eefdb6482..bc235c112c 100644
--- a/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
+++ b/src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs
@@ -27,13 +27,16 @@ internal sealed class ResponseCompressionBody : Stream, IHttpResponseBodyFeature
     private bool _autoFlush;
     private bool _complete;
 
+    private readonly bool _allowCompression;
+
     internal ResponseCompressionBody(HttpContext context, IResponseCompressionProvider provider,
-        IHttpResponseBodyFeature innerBodyFeature)
+        IHttpResponseBodyFeature innerBodyFeature, bool allowCompression = true)
     {
         _context = context;
         _provider = provider;
         _innerBodyFeature = innerBodyFeature;
         _innerStream = innerBodyFeature.Stream;
+        _allowCompression = allowCompression;
     }
 
     internal async Task FinishCompressionAsync()
@@ -224,6 +227,12 @@ internal sealed class ResponseCompressionBody : Stream, IHttpResponseBodyFeature
                 headers.Vary = StringValues.Concat(headers.Vary, HeaderNames.AcceptEncoding);
             }
 
+            // Only resolve compression provider when the request actually accepts compression
+            if (!_allowCompression)
+            {
+                return null;
+            }
+
             var compressionProvider = ResolveCompressionProvider();
             if (compressionProvider != null)
             {
diff --git a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
index 22a9677d6e..57ce180f3d 100644
--- a/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
+++ b/src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs
@@ -36,21 +36,20 @@ public class ResponseCompressionMiddleware
     /// <returns>A task that represents the execution of this middleware.</returns>
     public Task Invoke(HttpContext context)
     {
-        if (!_provider.CheckRequestAcceptsCompression(context))
-        {
-            return _next(context);
-        }
-        return InvokeCore(context);
+        // Always go through InvokeCore for all requests so that:
+        // 1. Vary: Accept-Encoding is added for compressible content types (for CDN/proxy caching)
+        // 2. Compression is only applied when Accept-Encoding is present
+        return InvokeCore(context, _provider.CheckRequestAcceptsCompression(context));
     }
 
-    private async Task InvokeCore(HttpContext context)
+    private async Task InvokeCore(HttpContext context, bool allowCompression)
     {
         var originalBodyFeature = context.Features.Get<IHttpResponseBodyFeature>();
         var originalCompressionFeature = context.Features.Get<IHttpsCompressionFeature>();
 
         Debug.Assert(originalBodyFeature != null);
 
-        var compressionBody = new ResponseCompressionBody(context, _provider, originalBodyFeature);
+        var compressionBody = new ResponseCompressionBody(context, _provider, originalBodyFeature, allowCompression);
         context.Features.Set<IHttpResponseBodyFeature>(compressionBody);
         context.Features.Set<IHttpsCompressionFeature>(compressionBody);
 

[dotnet-policy-service]

Thanks for your PR, @@kubaflo. Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

[Copilot]

Pull request overview

Updates ASP.NET Core’s response compression middleware so responses with compressible content types always include Vary: Accept-Encoding (even when the request lacks Accept-Encoding), improving HTTP cache correctness as described in #48008.

Changes:

• Always wrap responses in ResponseCompressionBody, while gating actual compression on whether the request accepts compression.
• Ensure Vary: Accept-Encoding is added for compressible responses regardless of request Accept-Encoding, and update middleware tests accordingly.
• Add multiple new .github/skills/* skill definitions, scripts, and bash test harnesses.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/Middleware/ResponseCompression/test/ResponseCompressionMiddlewareTest.cs	Updates expectations so Vary: Accept-Encoding is present even when Accept-Encoding is missing.
src/Middleware/ResponseCompression/test/ResponseCompressionBodyTest.cs	Updates unit tests to use the new ResponseCompressionBody constructor signature.
src/Middleware/ResponseCompression/src/ResponseCompressionMiddleware.cs	Always installs ResponseCompressionBody, passing an allowCompression flag based on request headers.
src/Middleware/ResponseCompression/src/ResponseCompressionBody.cs	Always appends Vary: Accept-Encoding for compressible responses; attempts compression only when allowed.
.github/skills/write-tests/SKILL.md	Adds documentation for a “write-tests” workflow skill.
.github/skills/verify-tests/SKILL.md	Adds documentation for verifying tests against buggy/fixed code.
.github/skills/try-fix/SKILL.md	Adds documentation for trying an alternative fix approach.
.github/skills/fix-issue/tests/test-skill-definition.sh	Adds bash tests intended to validate the fix-issue skill definition.
.github/skills/fix-issue/tests/test-ai-summary-comment.sh	Adds bash tests intended to validate AI summary comment scripts.
.github/skills/fix-issue/SKILL.md	Adds a large “fix-issue” skill definition/workflow document.
.github/skills/ai-summary-comment/scripts/post-ai-summary-comment.sh	Adds a script to post/update an “AI Summary” PR comment from local state files.
.github/skills/ai-summary-comment/SKILL.md	Adds documentation for the AI summary comment posting skill.