fix: use %#v instead of %q in Subset/NotSubset error messages#1848
fix: use %#v instead of %q in Subset/NotSubset error messages#1848veeceey wants to merge 6 commits intostretchr:masterfrom
Conversation
The %q format verb is intended for quoted strings and treats byte/int
slice elements as rune literals (e.g., '\x01' instead of 1). For types
that don't implement the fmt.Stringer interface with %q, it produces
broken output like %!q(bool=true).
This changes NotSubset (and the unsupported-type errors in Subset) to
use %#v, which produces proper Go-syntax representation for all types.
This is consistent with how Subset already formats its "does not
contain" failure messages.
Before: [%!q(bool=true)] is a subset of [%!q(bool=true)]
After: []bool{true} is a subset of []bool{true}
Fixes stretchr#1800
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Friendly ping - any chance someone could take a look at this when they get a chance? Happy to make any changes if needed. |
|
This sounds reasonable, but it's a known thing our test coverage on edge cases is low. So I wonder how this little change could affect existing code. But as it's only about the reporting message. I feel we are good. Maybe there is a need to limit the changes to one that were failing. So a method could be added to return the old %q for things that can be reported with it, and %#v for others. I might be wrong, and we are good, but I prefer to raise the point |
|
Thanks for the review @ccoVeille! Good call on scoping the changes — I can limit it to just the methods that were actually failing with the bad message. Want me to narrow it down? |
Revert %q -> %#v in "unsupported type" error paths for both Subset and NotSubset, keeping the fix only for the "is a subset of" and "could not be applied builtin len()" messages that were actually producing confusing output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Narrowed the scope to only the methods that were producing confusing output. Good call on keeping the blast radius small. |
Thanks for this change. Here again the commit shows off the lack of tests we have in testify. You were able to change this without having to change unit tests. Could you please add unit tests that are missing ? I assume you have a good knowledge of the types this method handles. |
|
Absolutely, I can add unit tests for the affected methods. You're right that changing the formatting without touching tests is a red flag for coverage gaps. I'll add tests that exercise the different data types and edge cases these methods handle to make sure the new formatting works as expected. |
Add TestSubsetNotSubsetErrorMessages to verify that error messages use %#v (Go-syntax) formatting across various data types: bool, float64, uint, byte, struct slices and maps. Each subtest also asserts that no broken %q artifacts (%!q) appear in the output.
Add tests for string slices (verifying %#v still readable), map-to-map with int keys, Subset failure with map of int keys, Subset failure with uint slices, and NotSubset with int32 slices. These cover additional code paths and type combinations to catch formatting regressions.
|
Added more unit tests to expand coverage for the error message formatting across different types:
All tests pass, including the full suite ( |
There was a problem hiding this comment.
Pull request overview
This PR fixes a bug in the NotSubset assertion error messages where the %q format verb was incorrectly used to format non-string collection types, producing broken output like %!q(bool=true) for booleans and '\x01' for integers. The fix changes the format verb from %q to %#v (Go-syntax representation), which already matches how the Subset function formats its "does not contain" failure messages.
Changes:
- Changed format verb from
%qto%#vin three locations within theNotSubsetfunction for "is a subset of" error messages - Updated test expectations in
TestSubsetNotSubsetto reflect the corrected Go-syntax format - Added comprehensive new test
TestSubsetNotSubsetErrorMessageswith 15 test cases covering various data types (bool, float64, uint, int32, byte, struct slices, and maps) - Updated test expectations in
TestNotSubsetWithSliceTooLongToPrintandTestNotSubsetWithMapTooLongToPrintfor truncated output
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| assert/assertions.go | Changed three error message format calls in NotSubset from %q to %#v to fix broken output for non-string types |
| assert/assertions_test.go | Updated test expectations to match new Go-syntax format and added comprehensive test coverage for error message formatting across multiple data types |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
assert/assertions.go
Outdated
There was a problem hiding this comment.
The format verb %q should be changed to %#v for consistency with the rest of the fix and to avoid broken output for non-string types. If a user passes a boolean or integer value where a slice/map is expected, this error message would display something like %!q(bool=true) has an unsupported type bool instead of a properly formatted error.
assert/assertions.go
Outdated
There was a problem hiding this comment.
The format verb %q should be changed to %#v for consistency with the rest of the fix and to avoid broken output for non-string types. If a user passes a boolean or integer value where a slice/map is expected, this error message would display something like %!q(bool=true) has an unsupported type bool instead of a properly formatted error.
assert/assertions.go
Outdated
There was a problem hiding this comment.
The format verb %q should be changed to %#v for consistency with the NotSubset fix and to avoid broken output for non-string types. If a user passes a boolean or integer value where a slice/map is expected, this error message would display something like %!q(bool=true) has an unsupported type bool instead of a properly formatted error.
assert/assertions.go
Outdated
There was a problem hiding this comment.
The format verb %q should be changed to %#v for consistency with the NotSubset fix and to avoid broken output for non-string types. If a user passes a boolean or integer value where a slice/map is expected, this error message would display something like %!q(bool=true) has an unsupported type bool instead of a properly formatted error.
assert/assertions.go
Outdated
There was a problem hiding this comment.
The format verb %q should be changed to %#v for consistency with the rest of the codebase and to avoid broken output for non-string types. If a user passes a boolean or integer value where a slice is expected, this error message would display something like %!q(bool=true) has an unsupported type bool, expecting array or slice instead of a properly formatted error. This is the same issue that was fixed in NotSubset's main error messages.
|
@veeceey could you check at Copilot feedbacks ? Maybe there is Jo need, but just in case. Please add tests if you changes some code |
Address Copilot review feedback: change the remaining %q format verbs to %#v in the "unsupported type" error paths of Subset, NotSubset, and isList. This prevents broken output like %!q(bool=true) when non-string types are passed where a slice/map is expected. Add TestSubsetNotSubsetUnsupportedTypes with 5 subtests covering each unsupported-type error path in Subset, NotSubset, and ElementsMatch.
| // TestSubsetNotSubsetErrorMessages verifies that Subset and NotSubset produce | ||
| // readable error messages using Go-syntax formatting (%#v) for various data types. | ||
| // This catches regressions where %q was used, which produces broken output like | ||
| // %!q(bool=true) for non-string types. |
There was a problem hiding this comment.
You should mention the related issues too
| Contains(t, msg, "[]bool{true} is a subset of []bool{true, false}") | ||
| // Ensure no broken %q formatting like %!q(bool=true) | ||
| NotContains(t, msg, "%!q") |
There was a problem hiding this comment.
I think this is enough
| Contains(t, msg, "[]bool{true} is a subset of []bool{true, false}") | |
| // Ensure no broken %q formatting like %!q(bool=true) | |
| NotContains(t, msg, "%!q") | |
| Contains(t, msg, "[]bool{true} is a subset of []bool{true, false}") |
The code may divert and this refers to something you remove, and that no longer exists once this PR will be merged.
For me checking, Contains is enough.
This comment apply to everything you added to this PR
There was a problem hiding this comment.
If we want to add a detection of bad formatting looking for "%!" would be OK if we consider it will detect:
- %!(MISSING)
- %!(EXTRA string=foo) and variation
- %!d(string=hello), %!q(bool=true)
But then I feel it should be part of testify test suite, and not limited to this PR, and the refactoring would be way larger than the scope of this PR and related issue
Summary
NotSubset(andSubset) error messages that use%qformat verb, which produces broken output for non-string types (e.g.,%!q(bool=true)for booleans,'\x01'for integers)%#v(Go-syntax representation) to match howSubsetalready formats its "does not contain" failure messagesBefore (broken):
After (fixed):
Fixes #1800
Test plan
TestSubsetNotSubsettests pass with updated expectationsTestNotSubsetNilpasses unchangedTestNotSubsetWithSliceTooLongToPrintpasses with updated expectationsTestNotSubsetWithMapTooLongToPrintpasses with updated expectationsTestSubsetWithSliceTooLongToPrintandTestSubsetWithMapTooLongToPrintpass unchangedgo test ./...) passes with no regressions🤖 Generated with Claude Code