Skip to content

adjust unit tests for test_save_load_float16 #12500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Nov 13, 2025
Merged

adjust unit tests for test_save_load_float16 #12500

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
b938d30
adjust unit tests for wan pipeline
kaixuanliu Oct 17, 2025
2244e23
update code
kaixuanliu Oct 30, 2025
906adf7
Merge branch 'main' into wan-pipeline
kaixuanliu Oct 30, 2025
5305169
avoid adjusting common `get_dummy_components` API
kaixuanliu Oct 30, 2025
a1d659c
Merge branch 'wan-pipeline' of https://github.com/kaixuanliu/diffuser…
kaixuanliu Oct 30, 2025
ecd4c8b
use `form_pretrained` to `transformer` and `transformer_2`
kaixuanliu Oct 30, 2025
62f3428
update code
kaixuanliu Nov 5, 2025
6b697ed
update
kaixuanliu Nov 5, 2025
91bdabf
Merge branch 'main' into wan-pipeline
sayakpaul Nov 5, 2025
3f2ab46
Merge branch 'main' into wan-pipeline
sayakpaul Nov 9, 2025
4bdfa35
Merge branch 'main' into wan-pipeline
DN6 Nov 10, 2025
b6e5a28
Merge branch 'main' into wan-pipeline
kaixuanliu Nov 11, 2025
6ec93a7
Merge branch 'main' into wan-pipeline
kaixuanliu Nov 12, 2025
1baf156
Merge branch 'main' into wan-pipeline
sayakpaul Nov 12, 2025
0ee299a
Merge branch 'main' into wan-pipeline
sayakpaul Nov 12, 2025
1a8dd43
Merge branch 'main' into wan-pipeline
sayakpaul Nov 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions tests/pipelines/test_pipelines_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1438,19 +1438,19 @@ def test_save_load_float16(self, expected_max_diff=1e-2):
with tempfile.TemporaryDirectory() as tmpdir:
pipe.save_pretrained(tmpdir)
pipe_loaded = self.pipeline_class.from_pretrained(tmpdir, torch_dtype=torch.float16)
for component in pipe_loaded.components.values():
for name, component in pipe_loaded.components.items():
if hasattr(component, "set_default_attn_processor"):
component.set_default_attn_processor()
pipe_loaded.to(torch_device)
if hasattr(component, "dtype"):
self.assertTrue(
component.dtype == torch.float16,
f"`{name}.dtype` switched from `float16` to {component.dtype} after loading.",
)
if hasattr(component, "half"):
# Although all components for pipe_loaded should be float16 now, some submodules still use fp32, like in https://github.com/huggingface/transformers/blob/v4.57.1/src/transformers/models/t5/modeling_t5.py#L783, so we need to do the conversion again manally to align with the datatype we use in pipe exactly
component = component.to(torch_device).half()
Copy link
Member

@sayakpaul sayakpaul Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem right at all. torch_dtype should be able to take care of it. I just ran it on my GPU for SD and it worked fine.

Copy link
Contributor Author

@kaixuanliu kaixuanliu Oct 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sayakpaul , I tested on A100, and when I print pipe_loaded.text_encoder.encoder.block[0].layer[1].DenseReluDense.wo.weight.dtype in L1455 , it returns torch.float32, not torch.float16, and the max_diff in L1456 is np.float16(0.0004883). When we apply this PR to align excatly with the behavior in pipe, the max_diff is 0. I think it's better to adjust the test case to make the output comparison of pipe and pipe_loaded apple to apple. WDYT?

Copy link
Member

@sayakpaul sayakpaul Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is torch_dtype in from_pretrained() should be enough for the model to be in fp16. Setting it with half() after loading the model in the FP16 torch_dtype seems erroneous to me.

I also ran the test on an A100, and it wasn't a problem. So, I am not sure if this test fix is correct at all.

Copy link
Contributor Author

@kaixuanliu kaixuanliu Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I printed pipe_loaded.text_encoder.encoder.block[0].layer[1].DenseReluDense.wo.weight.dtype after pipe_loaded = self.pipeline_class.from_pretrained(tmpdir, torch_dtype=torch.float16), and it returns torch.float32, it is root caused in L783, so I manualy add .half() to pipe_loaded, although it looks a bit wierd... On A100, the tolerance value is OK, but I think from the fundamentals perspective, the output from pipelines loaded from former saved should be exactly the same, that is the max_diff should be 0, right?

pipe_loaded.set_progress_bar_config(disable=None)

for name, component in pipe_loaded.components.items():
if hasattr(component, "dtype"):
self.assertTrue(
component.dtype == torch.float16,
f"`{name}.dtype` switched from `float16` to {component.dtype} after loading.",
)

inputs = self.get_dummy_inputs(torch_device)
output_loaded = pipe_loaded(**inputs)[0]
max_diff = np.abs(to_np(output) - to_np(output_loaded)).max()
Expand Down