Add sewing kit and utilities used for pruning scoring - pruning scoring is self-contained now #584

danielkorzekwa · 2025-11-20T15:47:48Z

What does this PR do?

Add sewing kit and utilities used for pruning scoring - pruning scoring is self-contained now - no dependency on internal Nvidia code.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

…ons_4 Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

…core_pruning_activations_5 Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

…core_pruning_activations_6sewing_kit

…korzekwa/score_pruning_activations_7

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

codecov · 2025-11-20T16:03:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.37%. Comparing base (f10be0d) to head (5a84ec9).
⚠️ Report is 1 commits behind head on feature/compress.

Additional details and impacted files

@@                Coverage Diff                @@
##           feature/compress     #584   +/-   ##
=================================================
  Coverage             74.37%   74.37%           
=================================================
  Files                   182      182           
  Lines                 18219    18219           
=================================================
  Hits                  13550    13550           
  Misses                 4669     4669

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

AAnoosheh · 2025-11-21T15:28:13Z

modelopt/torch/_compress/sewing_kit/passage/recipes/__init__.py

@@ -0,0 +1,15 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.


This folder is empty other than this?

Deleted this empty module. Note that later some content can be added here. Currently, we only add required modules from sewing_kit to support llama 3.1 8B ffn/att pruning scenario.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

kevalmorabia97

Minor comments. Feel free to merge after addressing these

kevalmorabia97 · 2025-11-24T12:34:11Z

modelopt/torch/_compress/sewing_kit/passage/core.py

+if sys.version_info[1] < 9:  # if less than pytorch 3.9
+    from typing import Sequence, Callable
+else:


modelopt only supports python 3.10+ so no need for if-else or from __future__ import annotations in any file

removing from __future__ import annotations will fail on self-referencing, e.g.,

modelopt/torch/_compress/sewing_kit/passage/core.py:57: in InputArgs def __add__(self, other: Any) -> InputArgs: E NameError: name 'InputArgs' is not defined

but I removed:

if sys.version_info[1] < 9: # if less than pytorch 3.9 from typing import Sequence, Iterable else:

removed from __future__ import annotations from tests/gpu/torch/export/test_fsdp2_export.py

modelopt/torch/_compress/utils/data/dataset.py

kevalmorabia97 · 2025-11-24T13:57:52Z

modelopt/torch/_compress/utils/validate_runtime_pipeline.py

All of this could be removed later by just using Nemo AutoModel's PP implementation

Created an internal NVidia issue to replace validate_runtime_pipeline.py with Nemo AutoModel's PP implementation: issues/70

kevalmorabia97 · 2025-11-24T13:59:31Z

modelopt/torch/_compress/utils/validation.py

+    # teacher_dir = (
+    #     '/lustre/fsw/portfolios/coreai/projects/coreai_nvfm_llm/models/'
+    #     'meta-llama/Meta-Llama-3.1-405B-Instruct/'
+    # )


please remove this

Co-authored-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Signed-off-by: Daniel Korzekwa <daniel.korzekwa@gmail.com>

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

danielkorzekwa added 30 commits November 3, 2025 21:05

Add decilm modelling code

694c317

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm modelling code.

991659f

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add transformers codebase

8489cee

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add transformers code

f0afefe

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm modelling code

b3ed5bc

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm modelling code

a700da5

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Correct licence headers

b59b679

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Correct licence headers

1abdf3e

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm code

66609b1

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm code

7da0a8a

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm code

6e09a81

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm code

2e3f5da

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add decilm code

418890e

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Make llama3 converter self-contained (no deps on internal Nvidia code)

01f4fc1

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add common module

c57eed4

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

module refactoring

3dc37b3

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

refactoring

10ffdfe

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add shared_checkpointing_utils

27a4456

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add json tools

b0e22b7

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add logger

52e7827

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

import refactoring

f5c1c87

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add post_init_sparse module

0aa6320

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add post_init_sparse

35d0dbc

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

merginy hydra.py and hydra_utils.py

e39a1ad

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add integrationt test for attention pruning

1bd0c67

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add score_pruning_activations

0ecd52b

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

import refactoring

278c6b7

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add dist_utils

7a0af16

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add validate_model

0f0cbbd

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Add activation scoring hooks for pruning

cb5cf25

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

danielkorzekwa added 10 commits November 6, 2025 09:48

add sewing_kit

ca7ab3f

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

add sewing_kit

a7a4adc

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

fix imports

ad84c26

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

fix imports

3d7e8a2

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Merge branch 'feature/compress' into dkorzekwa/score_pruning_activati…

6e96f81

…ons_4 Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Delete not needed tokenizer.

d541baa

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Merge branch 'dkorzekwa/score_pruning_activations_4' into dkorzekwa/s…

6cde527

…core_pruning_activations_5 Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Merge branch 'dkorzekwa/score_pruning_activations_5' into dkorzekwa/s…

9e32497

…core_pruning_activations_6sewing_kit

Merge branch 'dkorzekwa/score_pruning_activations_6sewing_kit' into d…

e4dd405

…korzekwa/score_pruning_activations_7

Fix imports

ae07708

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

danielkorzekwa requested review from a team as code owners November 20, 2025 15:47

danielkorzekwa requested a review from kevalmorabia97 November 20, 2025 15:47

fix imports

a319d33

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

kevalmorabia97 requested a review from AAnoosheh November 20, 2025 16:49

danielkorzekwa added 5 commits November 20, 2025 17:59

fix imports

a09e894

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Improve doc strings

c69c4a9

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Improve doc strings

1ee8026

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Improve doc strings.

d208f2d

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

Remove not used stuff from kd_model + add doc string

7a89dfa

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

AAnoosheh reviewed Nov 21, 2025

View reviewed changes

Remove empty module

09ac420

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

AAnoosheh approved these changes Nov 21, 2025

View reviewed changes

kevalmorabia97 approved these changes Nov 24, 2025

View reviewed changes

danielkorzekwa and others added 3 commits November 24, 2025 21:08

Update modelopt/torch/_compress/utils/data/dataset.py

22a3afb

Co-authored-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Signed-off-by: Daniel Korzekwa <daniel.korzekwa@gmail.com>

remove if-else check for pytorch <3.9

138d01e

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

code clean up

5a84ec9

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>

danielkorzekwa merged commit 194b532 into feature/compress Nov 24, 2025
21 checks passed

danielkorzekwa deleted the dkorzekwa/score_pruning_activations_7 branch November 24, 2025 23:02

		@@ -0,0 +1,15 @@
		# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Add sewing kit and utilities used for pruning scoring - pruning scoring is self-contained now #584

Add sewing kit and utilities used for pruning scoring - pruning scoring is self-contained now #584

Uh oh!

Conversation

danielkorzekwa commented Nov 20, 2025

What does this PR do?

Uh oh!

codecov bot commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevalmorabia97 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Nov 20, 2025 •

edited

Loading